The typo'ed spelling is only used in
_strip_additional_query_parameters(), which is as the name implies,
removes from the request any keys that are not present in the schema.
The nuance is that this happens only _after_ calling the
_schema_validation_helper() (which actually
validates the request against the schema), and _only_ if the
validation succeeds.
So while yes, _strip_additional_query_parameters() erroneously tried
to find 'addtionalProperties' in the schema, never found it, always
defaulted to True, and proceed to unconditionally strip any keys not
found in the schema, this was never a problem: the validation had
already been done, and if additionalProperties was False, the
validation correctly failed if extra keys were present.
Which means that if the code flow ever got to
_strip_additional_query_parameters() _with_ extra keys present in the
request, it means additionalProperties was True anyways, and it made
no material difference.
And I hate that I spent over an hour trying to understand why this
typo makes no difference in unit tests of behaviour.
Change-Id: I1ba5b9fa2ec5e4021543021068625d61ee671cdb
Node device XML validation flags [1]:
VIR_NODE_DEVICE_(CREATE|DEFINE)_XML_VALIDATE
were added in libvirt 8.10.0 but we support older libvirt versions
which will raise an AttributeError when flag access is attempted.
We are not currently using the flags (nothing calling with
validate=True) so this removes the flags from the code entirely. If the
flags are needed in the future, they can be added again at that time.
Closes-Bug: #2076163
[1] https://github.com/libvirt/libvirt/commit/d8791c3c7caa6e3cadaf98a5a2c94b232ac30fed
Change-Id: I015d9b7cad413986058da4d29ca7711c844bfa84
Libvirt 8.0.0 introduced the field to tell availability of TPM support,
which internally checks availability of swtpm[1]. We can use this
feature instead of implementing the own logic to detect availability.
The feature in libvirt relies on libvirt itself, so allows us to ensure
the swtpm binaries are present in the PATH used by libvirt, not in one
used by nova-compute.
In addition, libvirt 8.6.0 introduced the additional field to expose
available TPM versions[2], which can be used to report available TPM
versions.
<domainCapabilities>
<devices>
<tpm supported='yes'>
<enum name='model'>
<value>tpm-tis</value>
<value>tpm-crb</value>
</enum>
<enum name='backendModel'>
<value>passthrough</value>
<value>emulator</value>
<value>external</value>
</enum>
<enum name='backendVersion'>
<value>1.2</value>
<value>2.0</value>
</enum>
</tpm>
</devices>
</domainCapabilities>
[1] https://github.com/libvirt/libvirt/commit/6f7fc0b54ad97e62e10cd6f9524fcc29b4bb4cc5
[2] https://github.com/libvirt/libvirt/commit/1277a9c884039e92765c977917420511f45e52e8
Partially-Implements: blueprint libvirt-detect-vtpm-support
Change-Id: I9d07635883f413b8e91312f602a3f5de8f3bf266
Libvirt uses swtpm_ioctl to terminate swtpm processes. If the binary
does not exist, swtpm processes are kept running after the associated
VM terminates, because QEMU does not send shutdown to swtpm.
Closes-Bug: #2052761
Change-Id: I682f71512fc33a49b8dfe93894f144e48f33abe6
Note that this includes seemingly-unrelated test changes because we
were actually skipping the snapshot_running test for libvirt, which
has been a bug for years. In that test case, when we went to look
for image_meta.disk_format, that attribute was not set on the o.vo
object, which raised a NotImplementedError. That error is also checked
by the test to skip the test for drivers that do not support snapshot,
which meant that for libvirt, we haven't been running that case
beyond the point at which we create snapshot metadata and trip that
exception. Thus, once removing that, there are other mocks not in
place that are required for the test to actually run. So, this adds
mocks for qemu_img_info() calls that actually try to read the file on
disk, as well as the privsep chown() that attempts to run after.
Change-Id: Ie731045629f0899840a4680d21793a16ade9b98e
When we moved the qemu-img command in fetch_to_raw() to force the
format to what we expect, we lost the ability to identify and react
to situations where qemu-img detected a file as a format that is not
supported by us (i.e. identfied and safety-checked by
format_inspector). In the case of some of the other VMDK variants
that we don't support, we need to be sure to catch any case where
qemu-img thinks it's something other than raw when we think it is,
which will be the case for those formats we don't support.
Note this also moves us from explicitly using the format_inspector
that we're told by glance is appropriate, to using our own detection.
We assert that we agree with glance and as above, qemu agrees with
us. This helps us avoid cases where the uploader lies about the
image format, causing us to not run the appropriate safety check.
AMI formats are a liability here since we have a very hard time
asserting what they are and what they will be detected as later in
the pipeline, so there is still special-casing for those.
Closes-Bug: #2071734
Change-Id: I4b792c5bc959a904854c21565682ed3a687baa1a
These tests depend on qemu-img being installed and in the path, if it is not installed, skip them.
Change-Id: I896f16c512f24bcdd898ab002af4e5e068f66b64
Closes-bug: #2073862
Signed-off-by: Julien Le Jeune <julien.le-jeune@ovhcloud.com>
Make it extra clear this is only for API response bodies.
Change-Id: Id88c828aa7bcb1eff6dd149e8a92a19649ba4b0e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Kernels don't accept to access the governor strategy on an offline core, so
we need to only validate strategies for online cores.
Change-Id: I14c9b268d0b97221216bd1a9ab9e48b48d6dcc2c
Closes-Bug: #2073528
When the script was created there were only stable/* branches, but now
there are unmaintained/* branches as well, where the validator fails
when looking for hashes only on stable/* branches even if the given
hash is already on unmtaintained/* branch. This patch matches now both
stable/* and unmaintained/* branches.
Change-Id: I08fcc63ab0fbe5af1be70d5fde5af98bf006101c
This is a fix for the test whether a patch is bot generated or not, as
that did not worked as intended. The problem is that the script is
checking the email address of the parent patch (HEAD~), which probably
should be right in case the patch would be a MERGE patch. But this is
wrong in case the patch is not a MERGE patch. This fix uses the very
same pattern as it is using for the commit message parsing: the
$commit_hash variable, which is the parent's commit hash if the patch
is a MERGE patch, and an empty string in the other case (causing to
call 'git show' on HEAD).
Change-Id: I0abc72180edf34a6dd0624a40fb8682397805eca
Some version of mkisofs does not properly handle if both the input and
the output file of the command are the same. So this commit changes the
unit tests depending on that binary to use a different files.
Related-Bug: #2059809
Change-Id: I6924eb23ff5804c22a48ec6fabcec25f061906bb
Bug #2071972 brought it to light that we may have had some unit tests
improperly operating against lists, when generators are used by the SDK.
As a result of auditing remaining unit tests, this one instance was
found of improper testing. Fixing this should avoid future bugs like
2071972.
Related-Bug: #2071972
Change-Id: Icf9882f5f8421b150c308eb2502623f2fcb6cff6
while backporting Ia34203f246f0bc574e11476287dfb33fda7954fe
We observed that several of the tests showed distro specific
behavior depending on if qemu was installed in the test env,
what version is installed and how it was compiled
This change ensures that if qemu is present that it
supprot the required formats otherwise it skips the test.
Change-Id: I131996cdd7aaf1f52d4caac33b153753ff6db869
This change includes unit tests for the ISO
format inspector using mkisofs to generate
the iso files.
A test for stashing qcow content in the system_area
of an iso file is also included.
This change modifies format_inspector.detect_file_format
to evaluate all inspectors until they are complete and
raise an InvalidDiskInfo exception if multiple formats
match.
Related-Bug: #2059809
Change-Id: I7e12718fb3e1f77eb8d1cfcb9fa64e8ddeb9e712
This change adds a reproducer for the regression in iso
file support when
workarounds.disable_deep_image_inspection = False
Change-Id: I56d8b9980b4871941ba5de91e60a7df6a40106a8
When switching to using OpenStack SDK, there was a change missed
that didn't account for the SDK returning generators instead of
a list, so the loop on ports and port groups made it so that it
started returning an empty list afterwards.
Since there is no a masse of ports for a baremetal system usually,
we take the generator into a list right away to prevent this.
Closes-Bug: #2071972
Change-Id: I90766f8c225d834bb2eec606754107ea6a212f6d
This commit is a direct port of the format inspector
unit tests from glance as of commit
0d8e79b713bc31a78f0f4eac14ee594ca8520999
the only changes to the test are as follows
"from glance.common import format_inspector" was updated to
"from nova.image import format_inspector"
"from glance.tests import utils as test_utils"
was replaced with "from nova import test"
"test_utils.BaseTestCase" was replaced with "test.NoDBTestCase"
"glance-unittest-formatinspector-" was replaced with
"nova-unittest-formatinspector-"
This makes the test funtional in nova.
TestFormatInspectors requries qemu-img to be installed on the
host which would be a new depency for executing unit tests.
to avoid that we skip TestFormatInspectors if qemu-img
is not installed.
TestFormatInspectorInfra and TestFormatInspectorsTargeted
do not have a qemu-img dependency so
no changes to the test assertions were required.
Change-Id: Ia34203f246f0bc574e11476287dfb33fda7954fe
A Request-Too-Long error can be raised in case of too many aggregates
are attached to a host. The fix is to split the requests.
The request is executed by host via a periodic task. Result is set in
cache so no performances impact is expected.
The change introduce a new compute configuration option
`compute.sharing_providers_max_uuids_per_request = 200`.
Closes-bug: #2068893
Change-Id: I1355c0813cb99067ead60f9835938566289beb20
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
This restores the vmdk_allowed_types checking in create_image()
that was unintentionally lost by tightening the
qemu-type-matches-glance code in the fetch patch recently. Since we
are still detecting the format of base images without metadata, we
would have treated a vmdk file that claims to be raw as raw in fetch,
but then read it like a vmdk once it was used as a base image for
something else.
Change-Id: I07b332a7edb814f6a91661651d9d24bfd6651ae7
Related-Bug: #2059809
There is an additional way we can be fooled into using a qcow2 file
with a data-file, which is uploading it as raw to glance and then
booting an instance from it. Because when we go to create the
ephemeral disk from a cached base image, we've lost the information
about the original source's format, we probe the image's file type
without a strict format specified. If a qcow2 file is listed in
glance as a raw, we won't notice it until it is too late.
This brings over another piece of code (proposed against) glance's
format inspector which provides a safe format detection routine. This
patch uses that to detect the format of and run a safety check on the
base image each time we go to use it to create an ephemeral disk
image from it.
This also detects QED files and always marks them as unsafe as we do
not support that format at all. Since we could be fooled into
downloading one and passing it to qemu-img if we don't recognize it,
we need to detect and reject it as unsafe.
Change-Id: I4881c8cbceb30c1ff2d2b859c554e0d02043f1f5
It has been asserted that we should not be calling qemu-img info
on untrusted files. That means we need to know if they have a
backing_file, data_file or other unsafe configuration *before* we use
qemu-img to probe or convert them.
This grafts glance's format_inspector module into nova/images so we
can use it to check the file early for safety. The expectation is that
this will be moved to oslo.utils (or something) later and thus we will
just delete the file from nova and change our import when that happens.
NOTE: This includes whitespace changes from the glance version of
format_inspector.py because of autopep8 demands.
Change-Id: Iaefbe41b4c4bf0cf95d8f621653fdf65062aaa59
Closes-Bug: #2059809
Artom Lifshitz