Commit Graph

56584 Commits

Author SHA1 Message Date
Ghanshyam Mann ac68939cc5 Introduce scope_types in os-aggregates policy
oslo.policy introduced the scope_type feature which can
control the access level at system-level and project-level.
 - https://docs.openstack.org/oslo.policy/latest/user/usage.html#setting-scope
 - http://specs.openstack.org/openstack/keystone-specs/specs/keystone/queens/system-scope.html

Appropriate scope_type for nova case:
- https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/policy-defaults-refresh.html#scope

This commit introduce scope_type for os-aggregates API policies
as 'system'.

Also adds the test case with scope_type enabled and verify we
pass and fail the policy check with expected context.

Partial implement blueprint policy-defaults-refresh

Change-Id: Id920574fd7fa59f2a10e33dc458485bb4848347c
2020-03-06 10:02:25 +00:00
Zuul f176ffe6e5 Merge "Fix os-console-output policy to be admin_or_owner" 2020-03-05 21:32:59 +00:00
Ghanshyam Mann 036af931c9 Fix os-console-output policy to be admin_or_owner
os-console-output API policy is default to admin_or_owner[1] but API
is allowed for everyone.

We can see the test trying with other project context can access the API
- https://review.opendev.org/#/c/706724

This is because API does not pass the server project_id in policy target[2]
and if no target is passed then, policy.py add the default targets which is
nothing but context.project_id (allow for everyone who try to access)[3]

This commit fix this policy by passing the server's project_id in policy
target.

[1] https://github.com/openstack/nova/blob/1fcd74730d343b7cee12a0a50ea537dc4ff87f65/nova/policies/console_output.py#L27
[2] https://github.com/openstack/nova/blob/1fcd74730d343b7cee12a0a50ea537dc4ff87f65/nova/api/openstack/compute/console_output.py#L41
[3] https://github.com/openstack/nova/blob/c16315165ce307c605cf4b608b2df3aa06f46982/nova/policy.py#L191

Change-Id: I77759721138b9b4cc724895c8d15c1ccf2923995
Closes-bug: #1862558
2020-03-05 15:32:02 +00:00
Eric Fried f2d088b04e Stop using PlacementDirect
PlacementDirect was integrated into a functional test suite when it was
first created as a way to prove that it worked [1] and demonstrate how
to use it.

However, it was a pain then, because the interceptor needs to be created
every time you want to use it; and since extracted placement started
diverging from in-tree placement, other problems started cropping up
(see the associated bug).

So this commit removes the use of PlacementDirect from nova. Details:

- test_report_client now uses PlacementFixture. So all the `with
  interceptor` context management is gone. This accounts for the vast
  majority of the apparent change, which is just outdenting those
  contexts.
- SchedulerReportClientTestBase, which was doing some hocus pocus to
  wrap the SchedulerReportClient such that we could do some microversion
  checks, is removed. The test suite simply instantiates the
  microversion-checking wrapper class directly as the client used by the
  test cases.
- We were taking advantage of a PlacementDirect feature allowing us to
  default to the latest microversion if not explicitly specified in the
  request. Without this, we had to add the `version` kwarg to some of
  the calls we were making to SchedulerReportClient primitives
  (get/put/post/delete).
- A piece of test_update_from_provider_tree was using a
  deliberately-broken interceptor to prove that the code in question
  wasn't hitting the API. We replace this with a non-callable mock on
  the Adapter's request method.
- test_global_request_id was taking advantage of the interceptor to
  validate that the global request ID was making it to the "other side"
  of the API boundary. This was fun, but overkill. We now simply assert
  that the correct HTTP header is making it into the ksa Adapter's
  request method.
- Functional test suite test_resource_tracker.IronicResourceTrackerTest
  was inheriting from the SchedulerReportClientTestBase class, but not
  using the interceptor anywhere. Can't tell you why that was done. So
  now it just uses the plain old test.TestCase like everyone else.

[1] This commit does remove all of nova's testing of PlacementDirect.
However, it is still tested in the placement repository itself:
https://github.com/openstack/placement/blob/69b9659a457b6d715ed47e6bd6d3b923d548c620/placement/tests/functional/test_direct.py

Change-Id: Icb889c09a69e7c5cbf9330e5d9917d6ab3ac3dc5
Related-Bug: #1818560
2020-03-05 07:36:37 -06:00
Zuul 19cbbbebdd Merge "libvirt: Remove native LUKS compat code" 2020-03-05 04:44:42 +00:00
Zuul 1be4696c64 Merge "Introduce scope_types in os-agents policy" 2020-03-05 02:47:20 +00:00
Zuul 57f4634c40 Merge "Remove oslo_db.sqlalchemy.compat reference" 2020-03-05 02:28:23 +00:00
Zuul 1127bd0df6 Merge "functional: Avoid race and fix use of self.api within test_bug_1831771" 2020-03-04 23:33:54 +00:00
Zuul cfaa6ff3c5 Merge "Add test coverage of existing deferred_delete policies" 2020-03-04 21:01:10 +00:00
Zuul c521774e4a Merge "Fix os-os-deferred-delete policy to be admin_or_owner" 2020-03-04 21:01:01 +00:00
Zuul ce17fdc24c Merge "Pass the actual target in os-availability-zone policy" 2020-03-04 21:00:52 +00:00
Mike Bayer 91973ec26c Remove oslo_db.sqlalchemy.compat reference
oslo.db has removed this module in
I63e4baf772be9ddfb787ac3aff01fcaddf7b901c.  To the degree that
it is used in this Nova module is for a single check of SQLAlchemy
prior to 0.9.0 which is well below the minimum SQLAlchemy version
of 1.2.19.

Change-Id: I062da797ae4a4961535155a19ad330f5e8e14950
2020-03-04 16:59:48 +00:00
Lee Yarwood 649236bbb1 libvirt: Remove native LUKS compat code
I408baef12358a83921c4693b847a692f6c19e36f bumped the MIN versions of
Libvirt and QEMU past the required versions for native LUKS decryption
support during the Stein cycle.

As a result and building on I5a0de814f2868f1a4980a69b72b45ee829cedb94 we
can now remove various bits of compatibility code introduced to support
the use of native LUKS decryption in environments with mixed versions of
Libvirt and QEMU. This includes code used during N to N+1 upgrades that
is no longer required as both ends of a migration during an upgrade
should now have the required versions of Libvirt and QEMU.

An _is_luks_v1 utility method is retained to ensure the native approach
is only used when using LUKS v1 volumes as native LUKS v2 support is not
currently available within QEMU.

Change-Id: I41b7c1653c6a887ee4b08e588c5d422409aebfba
2020-03-04 15:49:43 +00:00
Lee Yarwood cbffac7df1 functional: Avoid race and fix use of self.api within test_bug_1831771
This test would previously only attempt to invoke a race between
instance.save(expected_task_state=task_states.SPAWNING) and a parallel
attempt to delete an instance when the instance also has a vm_state of
ACTIVE and task_state of None. However vm_state and task_state would
often be different within the test resulting in no attempt to invoke the
test being made.

As instance.save is only called with expected_task_state set to
task_states.SPAWNING by _unshelve_instance and _build_and_run_instance
we should just check for this and avoid any state races within the test.

Additionally when attempting to invoke the race this test would call
_wait_for_server_parameter and provide self.api. This change removes
this argument as since I8c96b337f32148f8f5899c9b87af331b1fa41424 this is
no longer required and will result in a `TypeError:
'TestOpenStackClient' object is not subscriptable` error.

Closes-Bug: #1866072
Change-Id: I36da36cc5b099174eece0dfba29485fc20b2867b
2020-03-04 13:39:45 +00:00
Ghanshyam Mann 8eeb8663b2 Add test coverage of existing deferred_delete policies
Current tests do not have good test coverage of existing policies.
Either tests for policies do not exist or if they exist then they
do not cover the actual negative and positive testing.

For Example, if any policy with default rule as admin only then
test should verify:
- policy check pass with context having admin role
- policy check fail with context having any other role than admin

As discussed in policy-defaults-refresh [1], to change the policies
with new default roles and scope_type, we need to have the enough
testing coverage of existing policy behavior.
When we will add the scope_type in policies or new default roles,
then these test coverage will be extended to adopt the new changes
and also make sure we do not break the existing behavior.

This commit covers the testing coverage of existing deferred_delete policies.

Partial implement blueprint policy-defaults-refresh

[1] https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/policy-defaults-refresh.html#testing

Change-Id: Ia84899b47f330ddb5993fc90e81d9817e8b9325a
2020-03-04 09:58:13 +00:00
Ghanshyam Mann f83c591e30 Fix os-os-deferred-delete policy to be admin_or_owner
os-deferred-delete restore server API policy is default to admin_or_owner[1] but API
is allowed for everyone.

We can see the test trying with other project context can access the API
- https://review.opendev.org/#/c/707455/

This is because API does not pass the server project_id in policy target[2]
and if no target is passed then, policy.py add the default targets which is
nothing but context.project_id (allow for everyone who try to access)[3]

This commit fix this policy by passing the server's project_id in policy
target.

Closes-bug: #1863009

[1] https://github.com/openstack/nova/blob/1fcd74730d343b7cee12a0a50ea537dc4ff87f65/nova/policies/deferred_delete.py#L27
[2] https://github.com/openstack/nova/blob/1fcd74730d343b7cee12a0a50ea537dc4ff87f65/nova/api/openstack/compute/deferred_delete.py#L38
[3] https://github.com/openstack/nova/blob/c16315165ce307c605cf4b608b2df3aa06f46982/nova/policy.py#L191

Change-Id: Ib05501b678d0b58bbd9e77cd5d79a9b6ef661497
2020-03-04 09:58:02 +00:00
Ghanshyam Mann c5bf8b230f Introduce scope_types in os-agents policy
oslo.policy introduced the scope_type feature which can
control the access level at system-level and project-level.
 - https://docs.openstack.org/oslo.policy/latest/user/usage.html#setting-scope
 - http://specs.openstack.org/openstack/keystone-specs/specs/keystone/queens/system-scope.html

Appropriate scope_type for nova case:
- https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/policy-defaults-refresh.html#scope

This commit introduce scope_type for os-agents API policies
as 'system'.

Also adds the test case with scope_type enabled and verify we
pass and fail the policy check with expected context.

Partial implement blueprint policy-defaults-refresh

Change-Id: If73cedb50185c830167db1c86ad15095c9f3d104
2020-03-04 09:48:03 +00:00
Ghanshyam Mann b055b5094e Add test coverage of existing os-console-auth-tokens policies
Current tests do not have good test coverage of existing policies.
Either tests for policies do not exist or if they exist then they
do not cover the actual negative and positive testing.

For Example, if any policy with default rule as admin only then
test should verify:
- policy check pass with context having admin or server owner
- policy check fail with context having non-admin and not server owner

As discussed in policy-defaults-refresh [1], to change the policies
with new default roles and scope_type, we need to have the enough
testing coverage of existing policy behavior.

When we will add the scope_type in policies or new default roles,
then these test coverage will be extended to adopt the new changes
and also make sure we do not break the existing behavior.

This commit covers the testing coverage of existing os-console-auth-tokens
policies.

Partial implement blueprint policy-defaults-refresh

[1] https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/policy-defaults-refresh.html#testing

Change-Id: Iface875e2af24fd2dced0daa079bf7595762e999
2020-03-04 09:47:06 +00:00
Ghanshyam Mann 4892607740 Pass the actual target in os-availability-zone policy
Currently if target is not passed in context.can(),
it use defauls target which is context.user_id, context.project_id.
These defaults target are not useful as it pass the
context's user_id and project_id only which means we tell
oslo policy to verify the context data with context data.

This commit pass the actual target for os-availability-zone policies
which is empty dict because policy rule is system scoped rather
than project, so the token scope check deals with the required
target checking.

Partial implement blueprint policy-defaults-refresh

Change-Id: I19fa9f2cb762baf5aeb5e9f25465863f9613f6db
2020-03-04 09:46:41 +00:00
Zuul 5aa8df8f5c Merge "Unplug VIFs as part of cleanup of networks" 2020-03-04 04:04:48 +00:00
Zuul 445d02314b Merge "Fix two test cases that use side effects in comprehensions" 2020-03-03 21:53:55 +00:00
Zuul d6450879c7 Merge "Remove 'nova.image.api' module" 2020-03-03 21:45:43 +00:00
Zuul 0f234d499a Merge "Pass the actual target in os-assisted_volume_snapshots policy" 2020-03-03 19:47:31 +00:00
Dan Smith 6dd73d191a Fix two test cases that use side effects in comprehensions
List comprehensions with side effects considered harmful. Fix two known
instances.

Change-Id: Idfc05de3b42d08b288ff611b5c87c27d837e1301
2020-03-03 07:23:40 -08:00
Zuul 72224775fb Merge "Monkey patch original current_thread _active" 2020-03-03 02:19:19 +00:00
Zuul f0df3c74c2 Merge "doc: note the need to configure cinder auth in reclaim_instance_interval" 2020-03-03 02:19:10 +00:00
Zuul 9728dff3a1 Merge "Add description of live_migration_timeout_action option" 2020-03-03 02:18:59 +00:00
Zuul ecb9e06b0d Merge "Workaround a race initialising version control in db_version()" 2020-03-03 02:18:50 +00:00
Zuul cbcf8183de Merge "api-ref: mark device response param as optional for list/show vol attachments" 2020-03-03 00:23:31 +00:00
Zuul 8cc54a4576 Merge "Add new default roles in os-assisted_volume_snapshots policies" 2020-03-03 00:23:20 +00:00
Zuul b84519034d Merge "Introduce scope_types in os-assisted_volume_snapshots policy" 2020-03-03 00:23:13 +00:00
Zuul ef297a7adc Merge "Add new default roles in Admin Action API policies" 2020-03-03 00:16:58 +00:00
Zuul ed09c2ced2 Merge "Add test coverage of existing os-assisted_volume_snapshots policies" 2020-03-02 20:48:26 +00:00
Zuul 3c0842cc51 Merge "Add test coverage of existing os-aggregates policies" 2020-03-02 20:30:20 +00:00
Zuul e0dd08c4ea Merge "Fix os-attach-interfaces policy to be admin_or_owner" 2020-03-02 19:33:24 +00:00
Zuul e1d61f6bf4 Merge "Define Cyborg ARQ binding notification event." 2020-03-02 17:25:54 +00:00
Zuul 0296e43a13 Merge "Add test coverage of existing os-agents policies" 2020-03-02 16:29:49 +00:00
Zuul ded3209058 Merge "Introduce scope_types in os-evacuate" 2020-03-02 15:40:08 +00:00
Zuul c8fb3f9612 Merge "Add test coverage of existing evacuate policies" 2020-03-02 14:28:02 +00:00
Zuul de22af97ab Merge "Add new default roles in os-availability-zone policies" 2020-03-02 14:27:49 +00:00
Zuul 2c91fccc0d Merge "Introduce scope_types in os-availability-zone" 2020-03-02 13:49:30 +00:00
Zuul f5f05516a2 Merge "Add test coverage of existing availability-zone policies" 2020-03-02 13:49:23 +00:00
Zuul 9460515205 Merge "Correct os-availability-zone policy check_str" 2020-03-02 13:49:16 +00:00
Zuul 7218e4a313 Merge "Add test coverage of existing admin_password policies" 2020-03-02 11:59:17 +00:00
Ghanshyam Mann fd1f464781 Add new default roles in Admin Action API policies
This adds new defaults roles in Admin Action API policies.

- All APIs are system admin role

Partial implement blueprint policy-defaults-refresh

Change-Id: I8b211b2345bf43c3605dc6c6d09bc2ce868783de
2020-03-02 11:54:04 +00:00
Ghanshyam Mann 7a5cc9b7b3 Pass the actual target in os-assisted_volume_snapshots policy
Currently if target is not passed in context.can(),
it use defauls target which is context.user_id, context.project_id.
These defaults target are not useful as it pass the
context's user_id and project_id only which means we tell
oslo policy to verify the context data with context data.

This commit pass the actual target for os-assisted_volume_snapshots
policies which is empty dict because policy rule is system scoped rather
than project, so the token scope check deals with the required
target checking.

Partial implement blueprint policy-defaults-refresh

Change-Id: I38393f3042adb82fef5e038c99296c27c88ed864
2020-03-02 11:52:52 +00:00
Ghanshyam Mann b017c22a84 Add new default roles in os-assisted_volume_snapshots policies
This adds new defaults roles in os-assisted_volume_snapshots policies. This policy
is default to system admin role.

Partial implement blueprint policy-defaults-refresh

Change-Id: I591d10f43fae40db9b8c222fcad32b77cac82e78
2020-03-02 11:48:03 +00:00
Ghanshyam Mann 3b729c448e Introduce scope_types in os-assisted_volume_snapshots policy
oslo.policy introduced the scope_type feature which can
control the access level at system-level and project-level.
 - https://docs.openstack.org/oslo.policy/latest/user/usage.html#setting-scope
 - http://specs.openstack.org/openstack/keystone-specs/specs/keystone/queens/system-scope.html

Appropriate scope_type for nova case:
- https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/policy-defaults-refresh.html#scope

This commit introduce scope_type for os-assisted_volume_snapshots API policies
as 'system'.

Also adds the test case with scope_type enabled and verify we
pass and fail the policy check with expected context.

Partial implement blueprint policy-defaults-refresh

Change-Id: I395ebec039f279521eca522e6e6dad1fb6653948
2020-03-02 11:47:19 +00:00
Ghanshyam Mann 3fd641d9ac Add test coverage of existing os-assisted_volume_snapshots policies
Current tests do not have good test coverage of existing policies.
Either tests for policies do not exist or if they exist then they
do not cover the actual negative and positive testing.

For Example, if any policy with default rule as admin only then
test should verify:
- policy check pass with context having admin role
- policy check fail with context having any other role than admin

As discussed in policy-defaults-refresh [1], to change the policies
with new default roles and scope_type, we need to have the enough
testing coverage of existing policy behavior.
When we will add the scope_type in policies or new default roles,
then these test coverage will be extended to adopt the new changes
and also make sure we do not break the existing behavior.

This commit covers the testing coverage of existing os-assisted_volume_snapshots
policies.

Partial implement blueprint policy-defaults-refresh

[1] https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/policy-defaults-refresh.html#testing

Change-Id: Ia5c683866c69a0b9824492b5e9c8d952ee18e347
2020-03-02 11:38:33 +00:00
Zuul 840defaaba Merge "libvirt: Provide the backing file format when creating qcow2 disks" 2020-03-02 11:28:54 +00:00