The 'nova.block_device' module is essentially a catchall utils-like
module for all things BDM. The 'get_bdm_image_metadata' module, and
closely related 'get_image_metadata_from_volume' both fall into the
category of functions that belong here so move them. This allows us to
clean up tests and, crucially, avoid a circular reference seen when we
want to use proper type hints in the 'nova.virt.driver' module.
nova.context imports...
nova.utils, which imports...
nova.block_device, which imports...
nova.virt.driver, which tries to import...
nova.context, causing a circular dependency
Change-Id: I48177d6e93f2ff132d26b53cd682fd24a43a4b31
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
To support new AVX512 cpu feature, this patch adds the mapping
between cpu features reported by libvirt [1] and corresponding
os-traits [2]. The traits include 'AVX512VBMI', 'AVX512IFMA',
'AVX512VBMI2', 'AVX512BITALG', 'AVX512VAES', 'AVX512GFNI',
'AVX512VPCLMULQDQ' and 'AVX512VPOPCNTDQ'.
[1] https://github.com/libvirt/libvirt/blob/master/src/cpu_map/x86_features.xml
[2] https://review.opendev.org/#/c/728576/
Depends-On: I58cba02f204d2e19a1a838852c628694edd47d73
Change-Id: I212a84e5cae15bc133aafd538d55f85149ab360f
mock does not accept 'autospec' and 'new' attributes
together to create mock object for tests. If autospec
is True the mock object is created using create_autospec()
and pass the keyword args to called function.
If both are passed then it start failing with oslotest 4.4.0
which is migrated to unittest.mock.
- https://d088a63e8edc60022939-485bd9eedc766dc9e54cbbb4001038bd.ssl.cf5.rackcdn.com/739330/1/check/cross-nova-py36/843ec44/testr_results.html
Creating mock with autospec only to fix this test to make
nova pass for oslo 4.4.0 so that requirement
can constraint oslotest to 4.4.0 (needed-by patch).
Change-Id: I9d6d97563114380f9a7549f08cede4b808352a3e
Needed-By: https://review.opendev.org/#/c/739531/
This is a minimal fix to workaround instance where libvirt
retruns stale data due to internal caching. In some cases
libivrt can return stale data vai the nodedev api when the
mac adress of an interface such as an sriov virtual function
canages, i.e. when a mac adress is reset after a vm with
a virtual funciton is migrated.
Change-Id: Ic5e60c8e28263365fad5867e483b6ad55cee7281
Partial-Bug: #1883671
In Ubuntu Bionic (18.04) mysql 5.7 version used to create
the user implicitly when using using the GRANT.
Ubuntu Focal (20.04) has mysql 8.0 and with mysql 8.0 there
is no implicit user creation with GRANT. We need to
create the user first before using GRANT command.
Nova unit and functional tests job using tools/test-setup.sh
script start failing when running on Ubuntu Focal
https://zuul.opendev.org/t/openstack/build/8b0f4fcc21854655a638c413b6fe1a91
Error log:
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 1064 (42000) at line 4: You have an error in your SQL syntax; check the manual
that corresponds to your MySQL server version for the right syntax to use near
'identified by 'openstack_citest' WITH GRANT OPTION' at line 2
Story: #2007865
Task: #40200
Change-Id: I97b0dcbb88c6ef7c22e3c55970211bed792bbd0d
Closes-Bug: #1885825
Execute glance upload in a native thread as it may block the current
coroutine until it completes.
Despite the fact we use eventlet monkey_patching [1] to achieve cooperative
yielding for network IO, file IO on busy file system may still get
nova-compute hanging.
Stick those IO in a native thread using eventlet tpool.execute() [2]
avoid this issue.
[1] https://eventlet.net/doc/patching.html
[2] https://eventlet.net/doc/threading.html
Closes-Bug: #1874032
Change-Id: I8dbc579e0037969aab4f2bb500fccfbde4190726
This change introduces new [DEFAULT]/max_concurrent_snapshots parameter
in order to limit by default compute resource overuse related
to snapshot.
Implements: blueprint max-concurrent-snapshots
Change-Id: I40b8caf06ed525e97e871cf381776164208461fb
It seems that with certain host configuration the ipv6 wsgi tests are
failing as the test only skipped if no ipv6 interface is available but
it does not check if the test will be able to bind to a port on that
interface. So this patch extends the skip criteria by checking if
binding is possible.
Change-Id: Iffea0586f02e8d4020cadc8c513bafe984e07a6e
Closes-Bug: #1883885
Cherry-pick check script validates the proposed patch's commit message.
If a patch is not on top of the given branch then Zuul rebases it to
the top and the patch becomes a merge patch. In this case the script
validates the merge patch's commit message instead of the original
patch's commit message and fails.
This fix selects the parent of the patch if it is a merge patch.
Change-Id: I8e4e5afc773d53dee9c1c24951bb07a45ddc2f1a
The libvirt driver reads under /proc and /sys and depends on os.stat to
detect if two directories are on the same device. So the driver is
heavily Linux dependent. This patch makes sure that the nova-compute
service refuse to start with the libvirt driver on a non Linux platform.
Change-Id: I29c5a678efec4fbc4bd7958ebe6d454ae30701cd
In Python 3.3+, 'sys.platform' no longer includes a major version [1]:
issue 12326: On Linux, sys.platform doesn't contain the major version
anymore. It is now always 'linux', instead of 'linux2' or 'linux3'
depending on the Linux version used to build Python. Replace
sys.platform == 'linux2' with sys.platform.startswith('linux'), or
directly sys.platform == 'linux' if you don't need to support older
Python versions.
Since we only care about Python 3.6 or greater, we could simply check
for 'linux'. However, the libvirt driver isn't supported on non-Linux
hosts so this was always a dumb check. Simply remove it instead.
[1] https://docs.python.org/3.3/whatsnew/3.3.html#porting-python-code
Change-Id: I207a6c8c8cf562477792c69e353ba99b3ec4cb5e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-Bug: #1882233
In Queens we deprecated the nova.image.download hook, which provided a
mechanism to inject custom code into the download path of our glance
module. This can now be removed.
The one known potential user of this is was a direct-rbd download
proposal, which needs further modification of the base glance client
in order to work. Thus, this removes the hook point, but not the
config option to allow specifying location schemes that should be
considered direct by the glance module. This provides a path forward
to integrate the direct-rbd code as a proper feature to our glance
module, which will mean un-deprecating that config option, but also
providing a stable list of potential options for it.
Change-Id: I7463af2ba9b74a73ffbb0a6b5fa12dff3fa5cac6
We have a race where if a user issues a delete request while an
instance is in the middle of booting, we could fail to update the
'queued_for_delete' field on the instance mapping with:
sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table
'instance_mappings' expected to update 1 row(s); 0 were matched.
This happens if we've retrieved the instance mapping record from the
database and then it gets deleted by nova-conductor before we attempt
to save() it.
This handles the situation by adding try-except around the update call
to catch StaleDataError and raise InstanceMappingNotFound instead,
which the caller does know how to handle.
Closes-Bug: #1882608
Change-Id: I2cdcad7226312ed81f4242c8d9ac919715524b48
This teaches libvirt's RBD image backend about the outside world, that
other ceph clusters may exist, and how to use Glance's multi-store image
import-via-copy mechanism.
The basic theory is that when we go to do the normal CoW clone for RBD,
we do the "does this image have a location that matches my RBD backend?"
check. If that check does not pass, if configured, we avoid failing
and ask Glance to copy it to our store instead. After that has completed,
we just recurse (once) and re-try our existing logic to see if the image
is now in a reachable location. If so, we pass like we would have
originally, and if not, we fail in the same way we would have.
The copy-to-store logic sets up a looping poll to check for copy completion
every N seconds according to a tunable, with a total timeout value in
case it never completes. If the timeout expires or Glance reports failure,
we will treat that the same as unreachable-due-to-location.
Related to blueprint rbd-glance-multistore
Change-Id: Ia839ad418b0f2887cb8e8f5ee3e660a0751db9ce
This just provides minimal support for calling the import API in Glance.
That API can do more things, but it is unlikely Nova would ever need to
call them, so this is rather opinionated and could be extended later if
needed.
Related to blueprint rbd-glance-multistore
Change-Id: Icf78fcabad8b966b6b5c289e1b660c01c928272d
When delete nova service by "nova service-delete <id>", this will delete the
compute node's resource provider in Placement. But if there are child resource
provider, Placement will throw exception(CannotDeleteParentResourceProvider)
and don't delete the resource provider. When we add the host service again,
Placement cannot insert new compute node resource provider, and we cannot use
the compute node any more. So we should delete sub resource provider in tree
when delete resource provider.
Modify unit test.
Change-Id: Ide8732be6c047ad1b141b89df676783b2fa2f25a
Closes-Bug: #1872385
The original fix of bug 1878024 missed an edge case where on a fresh
hypervisor the image cache directory hasn't been created yet. That
directory is only created when the first image is downloaded.
This patch makes sure that if the cache dir hasn't been created yet then
0 disk is reserved for the cache usage instead of raising and logging an
exception.
Change-Id: Id1bbc955a9099de1abc11b9063fe177896646d03
Related-Bug: #1878024
Closes-Bug: #1884214
Stephen Finucane