This patch removes the host parameter of the
notify_about_instance_delete() call as it is always filled with
CONF.host value.
Change-Id: Iff3b605b9410d5a097b53f532870df65780bc1e4
Implements: bp versioned-notification-transformation-stein
This patch moves soft_delete notification sending to the
notify_about_instance_delete context manager to unify the
delete notification sending code practice.
Implements: bp versioned-notification-transformation-stein
Change-Id: Ib6f95c22ffd3ea235b60db4da32094d49c2efa2a
The Iddbe50ce0ad3c14562df800bbc09ec5a7e840485 patch only considered
instance delete in the happy case when the instance is scheduled to a
compute successfully and the compute is available when the delete
action is executed. If the instance is never scheduled to a compute or
the compute is not available when the instance is deleted legacy delete
notifications are emitted from different places, compute.api instead of
compute.manager. The original patch missed these places.
There will be subsequent patch(es) handling the same edge cases
for soft_delete and force_delete.
Change-Id: If0693eab2ed31b5fbfe6cbafa5d67b69c2ed8442
Implements: bp versioned-notification-transformation-stein
The schema accepts the (resource provider) 'generation' key in the
resource provider section of the allocations dictionary in the POST
/allocations/{consumer_uuid} request payload. As with PUT
/allocations/{consumer_uuid}, it is ignored. This was missing from the
API reference.
Change-Id: I2586875a60aba7a566f6c1b61ded133b8749031c
This reverts commit bd7d991309 and bumps
the minimum version of oslo.db to 4.40.0, as that is the first version
of the library to include the renamed attribute.
Change-Id: Ic9e7864be3af7ef362cad5648dfc7bdecd104465
Related-Bug: #1788833
In commit 2d532963, all instances of 'async' were replaced with 'async_'
in preparation for Python 3.7. However, one of these should not have
been changed as it refers to an oslo.db object attribute. That attribute
has actually been renamed itself but that rename is only present from
oslo.db 4.40.0 [1]. Thankfully, an alias to the older name is provided
so we use that.
[1] https://github.com/openstack/oslo.db/commit/df6bf34
Change-Id: I1afd0ba34a9ebcb63edb91e880ef60580befb32e
Closes-Bug: #1788833
There was a eventlet.monkey_patch [1] when we launch a nova
process(like nova-api, nova-compute), but it's invalid under
the uwsgi mode.
But there are concurrency requirements in the api service, such
as, when listing instances cross multiple cells we're using
greenthreads and oslo.db does a time.sleep to allow switching
greenthreads [2].
So, in this patch we add the monkey_patch in the uwsgi
application setup and refactor the monkey patching to use common
code.
[1] https://github.com/openstack/nova/blob/233ea58/nova/cmd/__init__.py#L26
[2] https://github.com/openstack/oslo.db/blob/9c66959/oslo_db/sqlalchemy/engines.py#L51
Closes-bug: #1787331
Change-Id: Ie7bf5d012e2ccbcd63c262ddaf739782afcdaf56
We've seen a recent issue that suggest direct IO support checks can fail
in other valid ways than EINVAL, namely, failures with ENOENT or the
FileNotFoundError exception, which is a Python 3-only exception type,
can occur. While we can't test for this without breaking Python 2.7
support, we can mimic this by looking for checking for the errno
attribute of the OSError exception. Do this.
Change-Id: I8aab86bb62cbc8ad538c706af037a30437c7964d
Closes-Bug: #1788922
This patch modifies the code paths for the non-granular request group
allocation candidates processing. It removes the giant multi-join SQL
query and replaces it with multiple calls to
_get_providers_with_resource(), logging the number of matched providers
for each resource class requested and filter (on required traits,
forbidden traits and aggregate memebership).
Here are some examples of the debug output:
- A request for three resources with no aggregate or trait filters:
found 7 providers with available 5 VCPU
found 9 providers with available 1024 MEMORY_MB
found 5 providers after filtering by previous result
found 8 providers with available 1500 DISK_GB
found 2 providers after filtering by previous result
- The same request, but with a required trait that nobody has, shorts
out quickly:
found 0 providers after applying required traits filter (['HW_CPU_X86_AVX2'])
- A request for one resource with aggregates and forbidden (but no
required) traits:
found 2 providers after applying aggregates filter ([['3ed8fb2f-4793-46ee-a55b-fdf42cb392ca']])
found 1 providers after applying forbidden traits filter ([u'CUSTOM_TWO', u'CUSTOM_THREE'])
found 3 providers with available 4 VCPU
found 1 providers after applying initial aggregate and trait filters
Co-authored-by: Eric Fried <efried@us.ibm.com>
Closes-Bug: #1786519
Change-Id: If9ddb8a6d2f03392f3cc11136c4a0b026212b95b
Now that we have cell_uuid stashed on RequestContext, we can avoid
looking up the global marker again in the thread that matches
that cell. This removes one unnecessary lookup-by-value of the marker
value we already have and the associated FIXME.
Change-Id: Id1a0538de4cc3235680579ffa0ecf7f4eb24a1dd
This makes it easier to handle the cells that failed from layers
above, by reporting the cell uuids that reported, failed, and
timed out.
Related to blueprint handling-down-cell
Change-Id: Ied2264982a713bd27351eb690dd47f8531485d4f
This makes the instance_list module support batching across cells
with a couple of different strategies, and with room to add more
in the future.
Before this change, an instance list with limit 1000 to a
deployment with 10 cells would generate a query to each cell
database with the same limit. Thus, that API request could end
up processing up to 10,000 instance records despite only
returning 1000 to the user (because of the limit).
This uses the batch functionality in the base code added in
Iaa4759822e70b39bd735104d03d4deec988d35a1
by providing a couple of strategies by which the batch size
per cell can be determined. These should provide a lot of gain
in the short term, and we can extend them with other strategies
as we identify some with additional benefits.
Closes-Bug: #1787977
Change-Id: Ie3a5f5dc49f8d9a4b96f1e97f8a6ea0b5738b768
To check affinity and anti-affinity policies for scheduling instances,
we use the RequestSpec.instance_group.hosts field to check the hosts
that have group members on them. Access of the 'hosts' field calls
InstanceGroup.get_hosts during a lazy-load and get_hosts does a query
for all instances that are members of the group and returns their hosts
after removing duplicates. The InstanceList query isn't targeting any
cells, so it will return [] in a multi-cell environment in both the
instance create case and the instance move case. In the move case, we
do have a cell-targeted RequestContext when setup_instance_group is
called *but* the RequestSpec.instance_group object is queried early in
compute/api before we're targeted to a cell, so a call of
RequestSpec.instance_group.get_hosts() will result in [] still, even
for move operations.
This makes setup_instance_group query all cells for instances that are
members of the instance group if the RequestContext is untargeted, else
it queries the targeted cell for the instances.
Closes-Bug: #1746863
Change-Id: Ia5f5a0d75953b1154a8de3e1eaa15f8042e32d77
The time has come.
These filters haven't been necessary since Ocata [1]
when the filter scheduler started using placement
to filter on VCPU, DISK_GB and MEMORY_MB. The
only reason to use them with any in-tree scheduler
drivers is if using the CachingScheduler which doesn't
use placement, but the CachingScheduler itself has
been deprecated since Pike [2]. Furthermore, as of
change [3] in Stein, the ironic driver no longer
reports vcpu/ram/disk inventory for ironic nodes
which will make these filters filter out ironic nodes
thinking they don't have any inventory. Also, as
noted in [4], the DiskFilter does not account for
volume-backed instances and may incorrectly filter
out a host based on disk inventory when it would
otherwise be OK if the instance is not using local
disk.
The related aggregate filters are left intact for
now, see blueprint placement-aggregate-allocation-ratios.
[1] Ie12acb76ec5affba536c3c45fbb6de35d64aea1b
[2] Ia7ff98ff28b7265058845e46b277317a2bfc96d2
[3] If2b8c1a76d7dbabbac7bb359c9e572cfed510800
[4] I9c2111f7377df65c1fc3c72323f85483b3295989
Change-Id: Id62136d293da55e4bb639635ea5421a33b6c3ea2
Related-Bug: #1787910
This is a minor refactor with no functional change that will make
the next patch easier. If we're not limiting cell scope during an
instance list, we just let context.scatter_gather_all_cells() load
and use the global list of cells. Since we will want to count
those cells and adjust the batch size, we might as well just always
pass in the list of cells, which will either be a subset or all
of them.
Change-Id: Id167ab6d01698878d783999d54bcf77f37eec020
This extends the multi_cell_list module with batching support to avoid
querying N*$limit total results when listing resources across cells.
Instead, if our total limit is over a given threshold, we should query
smaller batches in the per-cell thread until we reach the total limit
or are stopped because the sort feeder has found enough across all cells
to satisfy the requirements. In many cases, this can drop the total number
of results we load and process from N*$limit to (best case) $limit+$batch
or (usual case) $limit+(N*$batch).
Since we return a generator from our scatter-gather function, this should
mean we basically finish the scatter immediately after the first batch query
to each cell database, keeping the threads alive until they produce all the
results possible from their cell, or are terminated in the generator loop
by the master loop hitting the total_limit condition. As a result, the
checking over results that we do immediately after the scatter finishes
will no longer do anything since we start running the query code for the
first time as heapq.merge() starts hitting the generators. So, this brings
a query_wrapper() specific to the multi_cell_list code which can mimic the
timeout and error handling abilities of scatter_gather_cells, but inline
as we're processing so that we don't interrupt the merge sort for a
failure.
Related-Bug: #1787977
Change-Id: Iaa4759822e70b39bd735104d03d4deec988d35a1
A guest must have a NUMA topology for numa-aware-vswitches to have any
effect. Call this out in the documentation.
Change-Id: Id0a637bcd0cbce29811acd7e56419350695cd3fd
This was always set by callers and appears to be here purely to avoid
updating tests. Update the tests and ease comprehension of the function.
Change-Id: Ib22b64ca499ffdb1a32d21f52240e80676ae1165
Balazs Gibizer