To observe the actual size of our pools and catch hanging threads in the
pools nova will log statistics of the executors each time a new work is
submitted but not more frequently than X seconds configurable via
[DEFAULT]thread_pool_statistic_period. But the default of the config is
-1 so the logging is disabled out of the box.
Most of the support should come from futurist but it isn't.
We can add them later there.
Change-Id: Id4244f5ae0fd49c99af2898789cdd510859e150d
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
Keystone project manager role can be used for the project-level
management APIs. Nova introduced the manager role in policy
defaults.
To introduced the manager role, we need to make migrations
policies more granular. Adding the separate policies for host
related operation allow us to open the migration operations
to project manager role. Existing policy is checked if migration
without specifying host and new policy is checked if host is
specified. Same will be applied to list migrations, new policy
will control to return the host info.
Also, Adding doc and releasenotes.
Partial implement blueprint policy-manager-role-default
Change-Id: Ie7d135e4d24ac6d53c46a4c69ade0b0bda554e71
Signed-off-by: Ghanshyam Mann <gmaan@ghanshyammann.com>
Signed-off-by: ghanshyam <gmaan@ghanshyammann.com>
... according to the TODO note added almost 5 years ago[1]. The method
is not used even now.
[1] fb14f24cc3
Change-Id: I2367680ca0f39cdfb5e0600ea67de254e744f4c3
Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
This helper is used only in unit tests, so should be maintained within
the tests module.
Change-Id: I1ce3dc4d7d27e3a8b3f1c525f721e40d79098af1
Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
Remove utility functions in nova.utils, which are not actually used
anywhere else in this repository.
Change-Id: I7c709f19bf3b44ed5695d9d44e6321eb7b5b0f8b
Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
This patch adds the logic to initialize our DEFAULT_EXECUTOR based on
the concurrency mode.
If the OS_NOVA_DISABLE_EVENTLET_PATCHING env variable is set to "true"
then we use ThreadPoolExecutor so the service will use native threads.
Otherwise the service will keep using the GreenThreadPoolExecutor and
therefore using Eventlet.
There is a new config option `[DEFAULT]default_thread_pool_size`
added to define the size of the ThreadPoolExecutor and the existing
`[DEFAULT]default_green_pool_size` is deprecated. The two config options
are independent and each of them is only used if the service is started
with the matching concurrency mode.
Our test fixture that catches leaked threads are adapted to handle both
threading mode. But at the moment it is only tested with the eventlet
mode as no service can be switched to threading mode yet.
Change-Id: I17b8065e8b14f0401297258e43dc2a98031d61f4
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
Now that we are using executors we can clarify the naming of the global
and the function accessing it.
Change-Id: Iddaa1e9c72f8f32168996edcfacf93d6df16e7a7
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
All our thread spawning should pass the context and all should go
through either spawn to use the default executor or spawn_on with a
specific executor. Right now pass_context is only called locally in the
utils module. We make it private to signal that it is intentional.
This is a pure refactor with two steps:
* remove the public pass_context as it is unused by removing it from the
internall call chain of spawn and spawn_on
* rename _pass_context_wrapper
This change does not effect any existing behavior so no test changes
were needed.
Change-Id: I5ad19b5c1a6d489c0f7df7fcbd9f87858b03760c
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
So far the ConductorManager.cache_images functionality used an
explicit GreenPool. We change this pool to a GreenThreadPoolExecutor
and use then spawn_on to start the thread. This way we can
* can replace the direct eventlet dependency with an indirect one behind
an interface where the impl can easily switched out to
ThreadPoolExecutor in the future
* we can get rid of the explicit pass_context call and centralize that
instead to spawn_*
We don't have unit test changes here as we have good unit tests that are
asserting behavior not asserting implementation. So when the implementation
changes (like in this refactor patch) but the behavior does not, we don't
need to change the existing unit test coverage.
The existing test coverage is in:
nova.tests.unit.conductor.test_conductor.ConductorTaskAPITest
Change-Id: I15297fd7869ed5f8a58f0ca599ef11df91e04de6
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
Small fixes for Ibff6c73ad9af911a42204e53fee31ed5537c829d
Change-Id: I288c48476a5cd6f0f657f29aab6246500fe52765
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
Yet another proxy API documented, albeit very loosely. We also remove a
conditional that can never be reached: we will always have a network
from neutron by time we attempt to show it. If we didn't, we'd have
exited early due to an exception.
Change-Id: I008975b3eabf5f3552ebad7e5bbe847b9c7eaa16
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
These are all empty and are purely here to satisfy the check for
schemas and to allow us to potentially populate them for documentation
purposes later.
Change-Id: Ia52bc78b3392ec69382f3427f5676d52f9abee6d
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
* Set additionalProperties to False, as expected
* Correct copy calls (not that it matters)
Change-Id: I97d8206d2df5deee0521ae69a73a32a7136c37be
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We also fix some issues in the api-ref, since it'll be another while
before we can replace that.
Change-Id: If661e3af796475637c0e76b3dfbfd5b7a7f38c24
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This patch adds just the objects and notifications required to
support an extra spec to configure a USB controller inside
the guest. This is useful for SPICE consoles using the native
protocol.
Change-Id: I791b16c5bf0e860a188783c863e95dc423998b0a
Signed-off-by: Michael Still <mikal@stillhq.com>
This is the implementation for the sound model extra spec as
desired by the new libvirt spice-direct console mode. Sound
support is a frequently requested feature for VDI users.
Change-Id: I33b8fc0136b4c1783b5c493e8ca9a15110767f6c
Signed-off-by: Michael Still <mikal@stillhq.com>
This patch adds just the objects and notifications required to
support an extra spec to configure a sound device inside
the guest. This is useful for SPICE consoles using the native
protocol.
Change-Id: I2faeda0fd0fb9c8894d69558a1ccaab8da9f6a1b
Signed-off-by: Michael Still <mikal@stillhq.com>
The bug is that the save code modifies the self.pci_devs.object list
while iterating it. This resulted in only half of the removed devs was
deleted at each save() run. Iterating on a shallow copy of that list
fixes it.
Closes-Bug: #2115729
Change-Id: I2711be9605618c1b93104d1dbddd8c7ee73b577e
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
This reproduces a bug where at each resource tracker
update_available_resources periodic run only half of the remove PCI
devices are deleted from the DB.
Related-Bug: #2115729
Change-Id: I95694f38105d61b4b7befd9223089f3a0d862634
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
So far the ComputeManager._sync_power_state functionality used an
explicit GreenPool. We change this pool to a GreenThreadPoolExecutor
and use then spawn_on to start the thread. This way we can
* can replace the direct eventlet dependency with an indirect one behind
an interface where the impl can easily switched out to
ThreadPoolExecutor in the future
* we can get rid of the explicit pass_context call and centralize that
instead to spawn_*
The ComputeManager live_migration_executor is already a
GreenThreadPoolExecutor so here we only switching to spawn_on to get rid
of the explicit pass_context call.
Change-Id: I51c5339d3f54f5c66856b6d4e06cd91ac04977cb
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
Now that spawn is using the default executor and returns a Future we can
have a spawn_on that takes the executor as a parameter so the
scatter-gather code path can use the same context passing logic but use
a separate executor.
This also change spawn() to use spawn_on() with the default executor.
And warns if the executor is busy an therefore the task is queued.
During testing this we discovered that SpawnIsSynchronous fixture made a
wrong assumption that eventlet.spawn(f) would raise the exception if f
raises. It does not. So now when we changed this fixture to the new
executor structure we fixed this test bug. There was two unit test
cases that depended on the wrong behavior they are adapted now.
Change-Id: I3bb40f5d2446dfd06253371d6abe8d3449b11265
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
As [1] switched over the implementation of spawn and spawn_n to the same
futurist Executor.submit we can now replace all spawn_n usage with spawn
and drop spawn_n from nova.utils.
[1]I3494660e1aaa1db46f9f08494cb5817ec7020cc5
Change-Id: I0027f119c0fbe8d5298307324eaf30c5e9e152d3
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
Nova uses nova.utils.spawn* to create new threads. This so far relied on
a GreenPool to provide GreenThreads. We changes this pool to a
futurist.GreenThreadPoolExecutor to have an interface where the
implementation can be swapped out to futurist.ThreadPoolExecutor to get
native threads instead.
This is an interface change on utils.spawn as it will return
futurist.Future instead of GreenThread. So couple of fixes needed across
nova to use:
* .result() instead of .wait()
* .add_done_callback() instead of .link(). Here we needed to change the
usage as the new callback does not forward args, so we rely on
closures instead.
This is also an interface and a behavior change for utils.spawn_n as it
now calls utils.spawn internally. This means that top of the above
detailed interface change there is behavior change for spawn_n.
The spawn creates GreenThread a wrapper around greenlet while spawn_n
created only the underlying greenlet. The greenlet cannot be managed
the same way as a more intelligent GreenThread, including the return
value but not limited to it, e.g. the whole cancellation mechanism
is missing from greenlet too. After this patch spawn_n will also use
GreenThread instead of naked greenlet. We consider the resulting small
performance change negligible.
Also the way we implement SpawnIsSynchronousFixture in our test is
adapted along with other test fixture adaptation to call / mock the
right functions.
Change-Id: I3494660e1aaa1db46f9f08494cb5817ec7020cc5
Signed-off-by: Balazs Gibizer <gibi@redhat.com>
In Ia5e4c6cadb6c88ccdf7e89566573f1f89087fbe5, we combined multiple
methods into one. However, we got the ranges off by one. Correct this.
We already remove some unnecessary version caps.
Change-Id: I6666101a636b963028fa12dd9caf3cba327bbe5e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Balazs Gibizer