Commit Graph

57256 Commits

Author SHA1 Message Date
zhangbailin 61b99a1295 Bump hacking min version to 3.1.0
This has introduced a new version of pycodestyle, which has improved its
parser to catch new corner cases and introduced some new tests.

Closes-Bug: #1878317
Change-Id: I853cf4dbd7ad6b7903a7f444f5c1da3f0fb21f77
2020-05-19 08:09:43 +08:00
Zuul 382d9b2336 Merge "Add link to PDF document" 2020-05-18 21:34:37 +00:00
Zuul 8283c54644 Merge "Make quotas respect instance_list_per_project_cells" 2020-05-18 21:14:19 +00:00
Zuul 574f6aa336 Merge "compute: Remove snapshot quiesce tests for STOPPED and SUSPENDED instances" 2020-05-18 19:06:32 +00:00
Zuul 9fefb4a9b8 Merge "compute: Allow snapshots to be created from PAUSED volume backed instances" 2020-05-18 19:03:54 +00:00
Andreas Jaeger 2eb2752988 Add link to PDF document
Switch to openstackdocstheme 2.2.0 that can link to PDF document,
enable this with setting openstackdocs_pdf_link.

Note that the link to the published document only works on 
docs.openstack.org where the PDF file is placed in the top-level html
directory. The site-preview places the PDF in a pdf directory.

Depends-On: https://review.opendev.org/728938
Change-Id: I29dddfa06183ecadb8f23feb2ad2e3c1c5a233cc
2020-05-18 18:42:31 +00:00
Zuul 3b4fa6108f Merge "tox: Integrate mypy" 2020-05-18 14:01:39 +00:00
Zuul b8fe7fac84 Merge "Switch to newer openstackdocstheme and reno versions" 2020-05-16 02:31:06 +00:00
Zuul 8b5d8b2c36 Merge "Update scheduler instance info at confirm resize" 2020-05-16 00:19:46 +00:00
Zuul a2bcaa1435 Merge "Reproduce bug 1869050" 2020-05-15 23:31:56 +00:00
Zuul 895334f0c7 Merge "docs: Add evacuation pre-conditions around the src host" 2020-05-15 23:07:32 +00:00
Mohammed Naser ab16946885 Make quotas respect instance_list_per_project_cells
This option was introduced in order to limit queries to cells which only
belong to the project we're interacting with.  However, The nova quota
code was not respecting it and therefore it was always trying to get the
cells assigned to a project even with that option disabled.

This patch makes the quota code respect that option and adds testing to
ensure that enabling the option does make sure it doesn't search all
cells but only the ones for this project.

Closes-Bug: #1878979
Change-Id: I2e0d48e799e70d550f912ad8a424c86df3ade3a2
2020-05-15 17:21:29 -04:00
Andreas Jaeger 6c963e7712 Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.1.2 and reno 3.1.0 versions. Using
these versions will allow parallelizing building of documents.

Update Sphinx version as well.

openstackdocstheme renames some variables, so follow the renames. A
couple of variables are also not needed anymore, remove them.

Set openstackdocs_auto_version to not version the documents.
Set openstackdocs_auto_name to use project as name.

Depends-On: https://review.opendev.org/728432
Change-Id: I4e3ae3ceabe125ea459ed4baabf2e98686268e50
2020-05-15 18:14:20 +02:00
Stephen Finucane 34ecf5ab91 tox: Integrate mypy
mypy is an experimental optional static type checker for Python that
aims to combine the benefits of dynamic (or "duck") typing and static
typing. While still in development, most features are supported and it's
already being used by real world projects like Sphinx. Let's start small
by integrating it into some of the interfaces that nova exposes. We can
further build upon this if it works out.

This change sets up the boilerplate necessary to use mypy in nova. Type
annotations are not included for any module - these will be added
separately. We're calling mypy by way of a script, as this allows us to
store a list of files that we have converted while we're in the process
of adding type annotations where necessary.

Change-Id: I75ab46a6768c4ca2050fdde2b7f8eeb90724c8c6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2020-05-15 15:59:53 +01:00
Lee Yarwood 7776cc02a1 compute: Remove snapshot quiesce tests for STOPPED and SUSPENDED instances
At present only the libvirt virt driver implements any form of instance
quiescing and this is only done when the instance is ACTIVE. As a result
we can remove these two tests for STOPPED and SUSPENDED instances.

Change-Id: Ie9db52c31bfe5df70456342c3ec7f83c0e23487f
2020-05-15 13:59:49 +00:00
Lee Yarwood cfde53e4b4 compute: Allow snapshots to be created from PAUSED volume backed instances
Iabeb44f843c3c04f767c4103038fcf6c52966ff3 allowed snapshots to be
created from PAUSED non-volume backed instances but missed the volume
backed use case.

This change simply adds PAUSED to the list of acceptable vm_states when
creating a snapshot from a volume backed instance in addition to the
already supported ACTIVE, STOPPED and SUSPENDED vm_states.

Closes-Bug: #1878583
Change-Id: I9f95a054de9d43ecaa50ff7ffc9343490e212d53
2020-05-15 13:59:42 +00:00
Zuul 575a91ff5b Merge "Bump hacking min version to 3.0.1" 2020-05-15 13:25:19 +00:00
Ghanshyam Mann 9f2e24a5a4 Bump hacking min version to 3.0.1
hacking 3.0.1 fix the pinning of flake8 to avoid bringing in a new
version with new checks.

bumping the min version for hacking so that any older hacking versions
which auto adopt the new checks are not used.

Depends-On: https://review.opendev.org/#/c/728335/

Change-Id: Ie00c10332bd7110169dbb150d601c157b6694d05
2020-05-15 02:36:37 +00:00
Zuul ef3b570732 Merge "Moving functional jobs to Victoria testing runtime" 2020-05-13 07:13:26 +00:00
Zuul 1d29344457 Merge "Suppress remaining policy warnings in unit tests" 2020-05-11 18:16:26 +00:00
Zuul 5b76ae4e66 Merge "Support for --force flag for nova-manage placement heal_allocations command" 2020-05-11 18:15:34 +00:00
Ghanshyam Mann 324c9b596b Suppress remaining policy warnings in unit tests
There are few place left in unit tests where policy warnings
are still logged.

- test_policy which is policy file tests and does policy
  initialization without suppressing the warnings.

- test_serversV21. PolicyFixture takes care of policy setup with
  no warning things[1] which is used by test base class[2] but
  test_serversV21 dulicate the policy initialization which leads
  to log warnings for unit tests. We do not need to initialization
  policy again and can reply on PolicyFixture setup.

  From the git history, it was added 7 years ago when no
  Fixture was available so there is no specific reason of re-initializating
  the policy in this test.
  - https://review.opendev.org/#/c/16160/3

[1] https://github.com/openstack/nova/blob/4b62c90063abc0c1b3e6b564a171e8c1d96cb735/nova/tests/unit/policy_fixture.py#L46
[2] https://github.com/openstack/nova/blob/4b62c90063abc0c1b3e6b564a171e8c1d96cb735/nova/test.py#L269

Change-Id: Ieb3f5510437d38bf2a4c8994d76c7f4001a6c9d8
2020-05-08 11:43:38 -05:00
Ghanshyam Mann c3834e675f Moving functional jobs to Victoria testing runtime
As per Victoria testing runtime[1], we need to tests py3.6,
py3.7, and py3.8.

- py3.7 is being tested with integration jobs.
- py3.6 and py3.8 are tested with unit test jobs.

Nova functional tests are testing py3.6 which can be moved to the latest
python available in this cycle which is 3.8. We do not need to run functional
or integration tests on each supported python version. Testing them on
the latest python version is enough. Coverage of all supported python version
is achieved via unit tests job.

[1] https://governance.openstack.org/tc/reference/runtimes/victoria.html

Change-Id: I1d6a2986fcb0435cfabdd104d202b65329909d2b
2020-05-08 11:01:34 -05:00
Zuul 7dfe999180 Merge "objects: Add migrate-on-load behavior for legacy NUMA objects" 2020-05-08 15:59:49 +00:00
Zuul 6a4e8b2467 Merge "config: Explicitly register 'remote_debug' CLI opts" 2020-05-08 11:14:32 +00:00
Zuul 9be9f2bc90 Merge "Wait for all servers to be active when testing vGPUs" 2020-05-08 09:11:25 +00:00
Zuul 487c4fa920 Merge "remove support of oslo.messaging 9.8.0 warning message" 2020-05-08 02:37:14 +00:00
Zuul 0d67cc9d5c Merge "Silence amqp heartbeat warning" 2020-05-08 02:32:31 +00:00
Sean Mooney ca2fe01cc1 remove support of oslo.messaging 9.8.0 warning message
This change removes support for supressing the
heartbeat message from olso.messaging before 9.8.0

Change-Id: I2035d5df31e43b730cd472cc438ec863bb538d62
Related-Bug: #1825584
2020-05-07 19:54:23 +01:00
Sean Mooney d6f664524d Silence amqp heartbeat warning
When the nova api is executing under uWSGI or MOD_WSGI
the lifetime of the amqp heartbeat thread is controlled
by the wsgi server. As a result when the nova api is run
in this configuration we expect that the heartbeat thread
will be suspended and heartbeats will be missed when the wsgi
server suspends execution of the wsgi application.

This change adds a python logging filter to suppress
the reporting of heartbeat warnings as this behavior is
expected. Since the operator cannot do anything to address
the issue the warning is just noise and many operators
and customers find it to be off-putting.

Change-Id: I642b1e3ed6de2be4dcc19fe214f84095d2e1d31a
Closes-Bug: #1825584
2020-05-07 18:47:27 +00:00
Sylvain Bauza dc5dc4de2f Wait for all servers to be active when testing vGPUs
We were just verifying the state of the first server before introspecting
the number of created devices. If the second server isn't ACTIVE yet,
then the mdevs wouldn't be found and the functest would return an exception.

Change-Id: I02a9f26dea6378281f3968a2c5cf0f652f9e342b
Closes-Bug: #1877281
2020-05-07 14:45:27 +02:00
Stephen Finucane 2362970d5c config: Explicitly register 'remote_debug' CLI opts
We were registering the 'remote_debug' CLI opts as part of the
'nova.conf' init code. This meant not only every service we used (bar
'nova-console') but also every CLI tool we provided was exposing these
options, even though they made zero sense in the latter option.

Resolve this by explicitly registering the options against the services
that we want them in. Specifically, these are the API services (nova-api,
nova-api-metadata, nova-api-os-compute), the console proxy services
(nova-novncproxy, nova-serialproxy, nova-spicehtml5proxy), nova-compute,
nova-conductor and nova-scheduler.

While we're here, we also clean up the documentation for these options
since there are a few non-rST'y things in there.

Change-Id: I7b7489e8412cc93d49904d4ef08a775b07b27d26
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2020-05-07 12:35:21 +00:00
Stephen Finucane 0f61d926b1 objects: Add migrate-on-load behavior for legacy NUMA objects
We started storing NUMA information as objects in the database in Kilo
(commit bb3202f859). Prior to this, we had stored this NUMA information
as a plain old dict. To facilitate the transition, we provided some
handlers based on these '_from_dict' functions. There were used to
ensure we could load old-style entries, converting them to objects which
we could later save back to the database.

It's been over four years (three since Kilo went EOL) and nine (nearly
ten) releases, meaning its time to look at dropping this code. At this
point, the only thing that could hurt us is attempting to do something
with a NUMA-based instance that hasn't been touched since they were
first booted on a Kilo or earlier host. Convert the '_to_dict'
functionality and overrides of the 'obj_from_primitive' method with a
similar check in the DB loading functions. Crucially, inside these DB
loading functions, save back when legacy objects are detected. This is
acceptable because the 'update_available_resource' in the resource
tracker pulls out both compute nodes and instances, with their embedded
fields like 'numa_topology', ensuring this will be run as part of the
periodic task.

NOTE: We don't need to worry about migrations of 'numa_topology' fields
in other objects: the 'RequestSpec' and 'MigrationContext' objects were
added in Liberty [1][2] and used the 'InstanceNUMATopology' o.vo from
the start, while the 'NUMATopology' object has only ever been used in
the 'ComputeNode' object.

Change-Id: I6cd206542fdd28f3ef551dcc727f4cb35a53f6a3
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2020-05-06 15:40:06 +01:00
Zuul 4b62c90063 Merge "Remove stale nested backport from InstancePCIRequests" 2020-05-06 14:03:54 +00:00
jay 87936baaac Support for --force flag for nova-manage placement heal_allocations command
Use this flag to forcefully heal allocation for a specific instance

Change-Id: I54147d522c86d858f938df509b333b6af3189e52
Closes-Bug: #1868997
2020-05-06 13:39:13 +02:00
Zuul 7d25b578da Merge "doc: Fix list rendering in cli/nova-status.rst" 2020-05-06 10:01:13 +00:00
Lee Yarwood 40e3366ea3 docs: Add evacuation pre-conditions around the src host
These were previously not listed but are important to re-enforce to
avoid corruption of instances during an evacuation.

Change-Id: Ia80dba0ade98940a5c4f94a9d2095e3c9ef21f97
2020-05-06 10:57:59 +01:00
Zuul 6ba06e23bc Merge "Follow-up for NUMA live migration functional tests" 2020-05-05 23:45:21 +00:00
Zuul 8461c9c834 Merge "Remove future imports" 2020-05-05 21:16:30 +00:00
Artom Lifshitz ca8f1f4222 Follow-up for NUMA live migration functional tests
This patch addresses outstanding feedback on
Ia3d7351c1805d98bcb799ab0375673c7f1cb8848 and
I78e79112a9c803fb45d828cfb4641456da66364a.

Change-Id: I70c4715de05d64fabc498b02d5c757af9450fbe9
2020-05-05 15:09:45 -04:00
Zuul dddc0dc825 Merge "Don't show upgr note for policy validation in V" 2020-05-05 13:50:06 +00:00
Zuul 9839e27880 Merge "Add nested resource providers limit for multi create" 2020-05-05 13:13:23 +00:00
Sylvain Bauza 505e1f1838 Don't show upgr note for policy validation in V
We merged Id9cd65877e53577bff22e408ca07bbeec4407f6e which included a reno
file after we branched Ussuri so we had to backport it to Ussuri but since
it was merged after RC1, the relnote would appear as an Upgrade section for
the Victoria release, which is not something we wanted (but rather for a
T->U upgrade).

Adding the ignore-notes section in order to avoid it.

Change-Id: If31933c5ff20167bf24fbad6f8746b1a9a6a36e5
2020-05-04 12:09:03 +02:00
Takashi Natsume 2b82575f3f doc: Fix list rendering in cli/nova-status.rst
Change-Id: I29ebb4c956ce979d60f74346d93510797f8de76b
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
2020-05-02 15:17:26 +09:00
Zuul fdcb1e306e Merge "Add nova-status upgrade check and reno for policy new defaults" 2020-05-02 03:44:15 +00:00
zhangbailin c19de075e1 Add nested resource providers limit for multi create
In 21.0.0 Ussuri we were completed the nova-cyborg interaction feature,
but there are some issue when multiple create instances.

Creating servers with accelerators provisioned with the Cyborg service,
if a flavor asks for resources that are provided by nested Resource
Provider inventories (eg. VGPU) and the user wants multi-create (ie. say
--max 2) then the scheduler could be returning a NoValidHosts exception
even if each nested Resource Provider can support at least one specific
instance, if the total wanted capacity is not supported by only one
nested RP.

For example,creating servers with accelerators provisioned with the
Cyborg service, if two children RP have 4 VGPU inventories each:
 - you can ask for a flavor with 2 VGPU with --max 2
 - but you can't ask for a flavor with 4 VGPU and --max 2

Related-Bug: #1874664
Change-Id: I64647a6ba79c47c891134cedb49f03d3c61e8824
2020-05-02 01:10:06 +00:00
Ghanshyam Mann d4af91f349 Add nova-status upgrade check and reno for policy new defaults
There are cases where policy file is re-generated freshly
and end up having the new defaults only but expectation is that
old deprecated rule keep working.

If a rule is present in policy file then, that has priority over
its defaults so either rules should not be present in policy file
or users need to update their token to match the overridden rule
permission.

This issue was always present when any policy defaults were changed
with old defaults being supported as deprecated. This is we have
changed all the policy for new defaults so it came up as broken case.

Adding nova-status upgrade check also to detect such policy file.

Related-Bug: #1875418

Change-Id: Id9cd65877e53577bff22e408ca07bbeec4407f6e
2020-05-01 15:32:51 +00:00
Zuul d078ecce0f Merge "NUMA LM: Add func test for bug 1845146" 2020-05-01 14:26:53 +00:00
Zuul 730eeb5fe0 Merge "Feature matrix: update AArch64 information" 2020-04-29 10:59:41 +00:00
Zuul ecc83108f0 Merge "libvirt:driver:Disallow AIO=native when 'O_DIRECT' is not available" 2020-04-29 10:59:35 +00:00