This change migrates the code currently written only to serve libvirt
driver to be generally useful, adding driver-neutral data structures
and a method to build them.
The libvirt driver is reworked to use get_instance_driver_metadata
instead of it's current code.
The ironic driver is reworked, per the blueprint, to send along some of
this additional metadata to the Ironic node.instance_info.
blueprint ironic-guest-metadata
Needed-By: https://review.opendev.org/c/openstack/ironic/+/924887
Change-Id: I2b23c8463f66c38e64625486157f245cd74cec61
Note that this includes seemingly-unrelated test changes because we
were actually skipping the snapshot_running test for libvirt, which
has been a bug for years. In that test case, when we went to look
for image_meta.disk_format, that attribute was not set on the o.vo
object, which raised a NotImplementedError. That error is also checked
by the test to skip the test for drivers that do not support snapshot,
which meant that for libvirt, we haven't been running that case
beyond the point at which we create snapshot metadata and trip that
exception. Thus, once removing that, there are other mocks not in
place that are required for the test to actually run. So, this adds
mocks for qemu_img_info() calls that actually try to read the file on
disk, as well as the privsep chown() that attempts to run after.
Change-Id: Ie731045629f0899840a4680d21793a16ade9b98e
When we moved the qemu-img command in fetch_to_raw() to force the
format to what we expect, we lost the ability to identify and react
to situations where qemu-img detected a file as a format that is not
supported by us (i.e. identfied and safety-checked by
format_inspector). In the case of some of the other VMDK variants
that we don't support, we need to be sure to catch any case where
qemu-img thinks it's something other than raw when we think it is,
which will be the case for those formats we don't support.
Note this also moves us from explicitly using the format_inspector
that we're told by glance is appropriate, to using our own detection.
We assert that we agree with glance and as above, qemu agrees with
us. This helps us avoid cases where the uploader lies about the
image format, causing us to not run the appropriate safety check.
AMI formats are a liability here since we have a very hard time
asserting what they are and what they will be detected as later in
the pipeline, so there is still special-casing for those.
Closes-Bug: #2071734
Change-Id: I4b792c5bc959a904854c21565682ed3a687baa1a
Make it extra clear this is only for API response bodies.
Change-Id: Id88c828aa7bcb1eff6dd149e8a92a19649ba4b0e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Kernels don't accept to access the governor strategy on an offline core, so
we need to only validate strategies for online cores.
Change-Id: I14c9b268d0b97221216bd1a9ab9e48b48d6dcc2c
Closes-Bug: #2073528
Some version of mkisofs does not properly handle if both the input and
the output file of the command are the same. So this commit changes the
unit tests depending on that binary to use a different files.
Related-Bug: #2059809
Change-Id: I6924eb23ff5804c22a48ec6fabcec25f061906bb
Bug #2071972 brought it to light that we may have had some unit tests
improperly operating against lists, when generators are used by the SDK.
As a result of auditing remaining unit tests, this one instance was
found of improper testing. Fixing this should avoid future bugs like
2071972.
Related-Bug: #2071972
Change-Id: Icf9882f5f8421b150c308eb2502623f2fcb6cff6
while backporting Ia34203f246f0bc574e11476287dfb33fda7954fe
We observed that several of the tests showed distro specific
behavior depending on if qemu was installed in the test env,
what version is installed and how it was compiled
This change ensures that if qemu is present that it
supprot the required formats otherwise it skips the test.
Change-Id: I131996cdd7aaf1f52d4caac33b153753ff6db869
This change includes unit tests for the ISO
format inspector using mkisofs to generate
the iso files.
A test for stashing qcow content in the system_area
of an iso file is also included.
This change modifies format_inspector.detect_file_format
to evaluate all inspectors until they are complete and
raise an InvalidDiskInfo exception if multiple formats
match.
Related-Bug: #2059809
Change-Id: I7e12718fb3e1f77eb8d1cfcb9fa64e8ddeb9e712
This change adds a reproducer for the regression in iso
file support when
workarounds.disable_deep_image_inspection = False
Change-Id: I56d8b9980b4871941ba5de91e60a7df6a40106a8
When switching to using OpenStack SDK, there was a change missed
that didn't account for the SDK returning generators instead of
a list, so the loop on ports and port groups made it so that it
started returning an empty list afterwards.
Since there is no a masse of ports for a baremetal system usually,
we take the generator into a list right away to prevent this.
Closes-Bug: #2071972
Change-Id: I90766f8c225d834bb2eec606754107ea6a212f6d
This commit is a direct port of the format inspector
unit tests from glance as of commit
0d8e79b713bc31a78f0f4eac14ee594ca8520999
the only changes to the test are as follows
"from glance.common import format_inspector" was updated to
"from nova.image import format_inspector"
"from glance.tests import utils as test_utils"
was replaced with "from nova import test"
"test_utils.BaseTestCase" was replaced with "test.NoDBTestCase"
"glance-unittest-formatinspector-" was replaced with
"nova-unittest-formatinspector-"
This makes the test funtional in nova.
TestFormatInspectors requries qemu-img to be installed on the
host which would be a new depency for executing unit tests.
to avoid that we skip TestFormatInspectors if qemu-img
is not installed.
TestFormatInspectorInfra and TestFormatInspectorsTargeted
do not have a qemu-img dependency so
no changes to the test assertions were required.
Change-Id: Ia34203f246f0bc574e11476287dfb33fda7954fe
A Request-Too-Long error can be raised in case of too many aggregates
are attached to a host. The fix is to split the requests.
The request is executed by host via a periodic task. Result is set in
cache so no performances impact is expected.
The change introduce a new compute configuration option
`compute.sharing_providers_max_uuids_per_request = 200`.
Closes-bug: #2068893
Change-Id: I1355c0813cb99067ead60f9835938566289beb20
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
This restores the vmdk_allowed_types checking in create_image()
that was unintentionally lost by tightening the
qemu-type-matches-glance code in the fetch patch recently. Since we
are still detecting the format of base images without metadata, we
would have treated a vmdk file that claims to be raw as raw in fetch,
but then read it like a vmdk once it was used as a base image for
something else.
Change-Id: I07b332a7edb814f6a91661651d9d24bfd6651ae7
Related-Bug: #2059809
There is an additional way we can be fooled into using a qcow2 file
with a data-file, which is uploading it as raw to glance and then
booting an instance from it. Because when we go to create the
ephemeral disk from a cached base image, we've lost the information
about the original source's format, we probe the image's file type
without a strict format specified. If a qcow2 file is listed in
glance as a raw, we won't notice it until it is too late.
This brings over another piece of code (proposed against) glance's
format inspector which provides a safe format detection routine. This
patch uses that to detect the format of and run a safety check on the
base image each time we go to use it to create an ephemeral disk
image from it.
This also detects QED files and always marks them as unsafe as we do
not support that format at all. Since we could be fooled into
downloading one and passing it to qemu-img if we don't recognize it,
we need to detect and reject it as unsafe.
Change-Id: I4881c8cbceb30c1ff2d2b859c554e0d02043f1f5
It has been asserted that we should not be calling qemu-img info
on untrusted files. That means we need to know if they have a
backing_file, data_file or other unsafe configuration *before* we use
qemu-img to probe or convert them.
This grafts glance's format_inspector module into nova/images so we
can use it to check the file early for safety. The expectation is that
this will be moved to oslo.utils (or something) later and thus we will
just delete the file from nova and change our import when that happens.
NOTE: This includes whitespace changes from the glance version of
format_inspector.py because of autopep8 demands.
Change-Id: Iaefbe41b4c4bf0cf95d8f621653fdf65062aaa59
Closes-Bug: #2059809
Tempest currently defaults to disk_formats[0] for images it creates,
which is 'ami'. However, it's actually using a qcow2 disk image by
default, which means we're lying to glance when we create those.
Change-Id: I737e9aa51c268a387f1eed24cf717618d057d747
The 'rxtx_factor' is a float, not an integer, and should allow values <
1 so long as they are > 0. Correct this.
Change-Id: I3567f13d7a1bb6f42755e3f5739fa9ecd827aa49
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We have no schemas yet and thus nothing to use this, but we enable it in
our tests for when we start adding these.
Change-Id: I160ee724459403a10516a356ee860831545d7d65
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Jay Faulkner