CLI options must be registered before the command line is parsed or a
cfg.ArgsAlreadyParsedError is raised.
Since commit 2cbea24 we are registering two 'remote_debug' arguments
in nova.service. In commit b506529, we see a hack being added to
ensure that nova.service is imported and the CLI opts registered
before config.parse_args() is called.
To clean this up somewhat, refactor all the elements of this remote
debugger support into nova.debugger. Avoid importing modules at
module import time since we need to import this before monkey
patching. Add a function for registering the CLI options just before
config.parse_args() is called. And fail gracefully if the CLI options
aren't registered when we called debugger.init().
Note that because the options aren't registered at module import time
any more, the config generator doesn't pick them up. However, that's
actually a good thing since this support doesn't work if you enable it
via config files since debugger.enabled() only looks at the command
line.
DocImpact: remote_debug options are not available via the config file,
only via the command line.
Change-Id: I97f747a2fb9222137203657df1d86ba89f3219e2
This file can be installed to properly configure the
firewall to allow VNC access to openstack instances
DocImpact
Closes Bug: 1294411
Change-Id: Ifc9a7d88c71dfe1b66d6ac395d9574cc619c1d3a
Added missing imports along with adding a call to config.parser_args as needed
to allow vm_vdi_cleaner to work against current master.
Corrected spelling error in comment.
Change-Id: I4d30d0f7f9718cb29fb8c4c18e9e6fa23f3496ef
Closes-Bug: #1279942
We don't need to have the vi modelines in each source file,
it can be set in a user's vimrc if required.
Also a check is added to hacking to detect if they are re-added.
Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6
Closes-Bug: #1229324
The oslo.messaging library takes the existing RPC code from oslo and
wraps it in a sane API with well defined semantics around which we can
make a commitment to retain compatibility in future.
The patch is large, but the changes can be summarized as:
* oslo.messaging>=1.3.0a4 is required; a proper 1.3.0 release will be
pushed before the icehouse release candidates.
* The new rpc module has init() and cleanup() methods which manage the
global oslo.messaging transport state. The TRANSPORT and NOTIFIER
globals are conceptually similar to the current RPCIMPL global,
except we're free to create and use alternate Transport objects
in e.g. the cells code.
* The rpc.get_{client,server,notifier}() methods are just helpers
which wrap the global messaging state, specifiy serializers and
specify the use of the eventlet executor.
* In oslo.messaging, a request context is expected to be a dict so
we add a RequestContextSerializer which can serialize to and from
dicts using RequestContext.{to,from}_dict()
* The allowed_rpc_exception_modules configuration option is replaced
by an allowed_remote_exmods get_transport() parameter. This is not
something that users ever need to configure, but it is something
each project using oslo.messaging needs to be able to customize.
* The nova.rpcclient module is removed; it was only a helper class
to allow us split a lot of the more tedious changes out of this
patch.
* Finalizing the port from RpcProxy to RPCClient is straightforward.
We put the default topic, version and namespace into a Target and
contstruct the client using that.
* Porting endpoint classes (like ComputeManager) just involves setting
a target attribute on the class.
* The @client_exceptions() decorator has been renamed to
@expected_exceptions since it's used on the server side to designate
exceptions we expect the decorated method to raise.
* We maintain a global NOTIFIER object and create specializations of
it with specific publisher IDs in order to avoid notification driver
loading overhead.
* rpc.py contains transport aliases for backwards compatibility
purposes. setup.cfg also contains notification driver aliases for
backwards compat.
* The messaging options are moved about in nova.conf.sample because
the options are advertised via a oslo.config.opts entry point and
picked up by the generator.
* We use messaging.ConfFixture in tests to override oslo.messaging
config options, rather than making assumptions about the options
registered by the library.
The porting of cells code is particularly tricky:
* messaging.TransportURL parse() and str() replaces the
[un]parse_transport_url() methods. Note the complication that an
oslo.messaging transport URL can actually have multiple hosts in
order to support message broker clustering. Also the complication
of transport aliases in rpc.get_transport_url().
* proxy_rpc_to_manager() is fairly nasty. Right now, we're proxying
the on-the-wire message format over this call, but you can't supply
such messages to oslo.messaging's cast()/call() methods. Rather than
change the inter-cell RPC API to suit oslo.messaging, we instead
just unpack the topic, server, method and args from the message on
the remote side.
cells_api.RPCClientCellsProxy is a mock RPCClient implementation
which allows us to wrap up a RPC in the message format currently
used for inter-cell RPCs.
* Similarly, proxy_rpc_to_manager uses the on-the-wire format for
exception serialization, but this format is an implementation detail
of oslo.messaging's transport drivers. So, we need to duplicate the
exception serialization code in cells.messaging. We may find a way
to reconcile this in future - for example a ExceptionSerializer
class might work, but with the current format it might be difficult
for the deserializer to generically detect a serialized exception.
* CellsRPCDriver.start_servers() and InterCellRPCAPI._get_client()
need close review, but they're pretty straightforward ports of code
to listen on some specialized topics and connect to a remote cell
using its transport URL.
blueprint: oslo-messaging
Change-Id: Ib613e6300f2c215be90f924afbd223a3da053a69
Commits included:
e3dddd7 generator: use EXTRA_* env vars in the bash script
e8e636c generator: add an EXTRA_LIBRARIES env variable
6da13e8 generator: rename EXTRA_MODULES_FILE to RC_FILE
5dce17b Use entry points to discover options in libraries
dd9aa2b Remove unused variables
ad17a69 Fix filter() usage due to python 3 compability
343686b Add check_uptodate to tools/config
The library option discovery support is the main addition we're
interested in since it is required for oslo.messaging.
Change-Id: I68958ddd1c1d0c02b54002c36b97270efaaaa892
Sync from oslo-incubator with the following patches about the config
generator:
7c18261 Temporary workaround for config generator
4b3e32b Sort the output of config/generator.py by group name
806272e Improve error reporting on import failures
6cd1c33 Clean up extra modules code
3d077a3 Print the deprecated group/name for each option
aec6d94 config.generator: allow extra modules importing
ddc4f0d Fix error when no custom config file matches *.conf.sample
d4bf84d Allow generate_sample MODULEPATH to be set in env
UpgradeImpact
DocImpact, the default values in keystone_authtoken config group are
changed.
Change-Id: If7181e14db2e43e42255bb6221d0543c13c4e5a6
Updates our example config file so that it includes configuration
settings for the auth_token middleware.
Includes the latest generate_sample.sh from Oslo.
Related-Bug: #1240753
Change-Id: I393a67f1f005e775928130c9241aa7e25c391ae3
This gets rid of a SQLAlchemy deprecation warning when using this tool w/
PostgreSQL:
The SQLAlchemy PostgreSQL dialect has been renamed from 'postgres' to
'postgresql'. The new URL format is
postgresql[+driver]://<user>:<pass>@<host>/<dbname>
This appears to have been deprecated in the SQLAlchemy 0.6 series so I think we
are Okay to remove it now.
Change-Id: If8fdf822ba452b387e9f2f3ec7c9157e4ae932f9
Updates the generate_samples.sh script so that we remove
.pyc files in the nova source tree before generating the sample file.
If you don't run unit tests the .pyc files won't get regenerated.
This can cause cryptic errors if you run tests, switch branches,
and then try to regenerate your config files if there are module
differences, etc.
Change-Id: I784a22c1cd7fb2c69ea1d9e02f87bd7da19d76aa
See I62ce43a330d7ae94eda4c7498782a655e63747fa for the gorey details on
why this exists.
As of this fix:
https://github.com/eventlet/eventlet/pull/34
which was released in eventlet 0.13, we no longer need the patch.
This has now been removed from oslo-incubator, so this is really just
syncing that removal.
Change-Id: I84267f3c6726cb2e750f615e107c48b12c6ed353
This is for Python 3 compatibility, since Python 3 does not support
the print statement, only the print function. (Python 2.6 and 2.7
support both, depending on whether a __future__ import is used.)
Paths containing "xen" were left alone, because of comments in a
couple of the files indicating that the XenServer code needed to be
compatible with Python 2.4.
Fixes bug 1226943
Change-Id: I23b804a4d99500b4acf81dd19645ab06dfcc9f1c
This patch reverts 13d80020. The help message should be added
in oslo-incubator since generate_sample.sh is synced from there also.
Resolves bug #1204204
9f2b93b Sample config file generator clean up
33f9f78 Replaces relative path to absolute path in conf file generator
c97e17b Refactors if statement in config generator
8b11fda Provide useful defaults for config generator
4c02e0a Unset OS_xx variable before generate configuration
Change-Id: I96fcb39c0c620c147fea5c96416e506a758619ce
Needed to make run_tests.sh identify requirements correctly
We need to check requirements.txt and test-requirements.txt
in parallel to ensure both are satisfied and we don't upgrade
a package beyond requirements.txt's limits when installing
dependencies for test-requirements.txt
Fixes bug: 1227019
Change-Id: I606f8a1d3eed5c270d5b3c154994d5c3a17b98c8
This just adds a sample command line for how to generate the
nova.conf.sample because I keep forgetting how to do it.
Change-Id: Id8bfb859f90d7fde51dd1ffdb40ec383ed5193e6
The sample command line lacks the arguments required for executing
the script. Added the command line used in check_uptodate.sh
Fixes bug #1204204
Change-Id: I475491872577632c472695a3b1089caaf1f27d7c
Previous _ was monkey patched into builtins whenever
certain modules were imported. This removes that and
simply imports it when it is needed.
Change-Id: I0af2c6d8a230e94440d655d13cab9107ac20d13c
First time run of the console rotate script failed, due to the fact that
the logs did not exist at that time.
Fixes bug 1204440
Change-Id: I49b5569c1ba3349d911caace42685fa0f19a1f66
The sample generator tool in Oslo is located in tools/config therefore
this patch moves the files in tools/conf to tools/config. The reason
Oslo chose 'config' is because the generator.py module is located in
the openstack.common.config package and update.py looks for the
corresponding directory in tools when it syncs the 'config' package.
./update.sh --module config --base nova --dest-dir $NOVA_HOME --nodep
Since every thing moved to tools/config, changes had to happen on
the tox.ini file and check_update.sh
Change-Id: Icbd467888cc7f16fa3694ed2b93548d0285461b5
We added this check recently to Ceilometer, and it helps a lot to keep this
file always up to date and not falling behind.
Change-Id: I5ad2c6366032a39c2e55dd11a16b9f1780ab9ed0
Includes these commits:
3948aee Highlighting the deprecated nature of 'log-format'.
b21fc56 Fix bad default for show_deleted
d28fa69 python3: Add python3 compatibility.
0bf03b7 Add network_utils.urlsplit
e456727 Remove useless logging in networks_utils
7119e29 Enable hacking H404 test.
b41862d Use param keyword for docstrings
2f01388 Use Python 3.x compatible except construct
e3545f8 Enable hacking H402 test
484a1df Enable hacking H403 test
35660da Enable hacking H401 test
874249e Add support for requirements.txt.
b135234 Remove the notifier and its dependencies from log.py
926b3e9 Fixes import order nits
Changes which may look not look so harmless:
- the common get_admin_context() method is not used by Nova
- network_utils.urlsplit() is added but not used yet
- suppport for requirements.txt is added to patch_tox_venv but
actually it doesn't use those files
Change-Id: I03e67f4648dcaf57620f11e63cfd8e7dbe3665cb
We already mostly use:
from nova import db
instead of:
import nova.db.api
So change remaining few cases to use nova.db instead of nova.db.api.
Help make code more uniform and easy to work with.
Change-Id: I6183352c4bbe77b88a5b277788f588b0b48c6476
Xen will produce the guest logs in a specified directory.
This scripts configures the above logging as expected
by the xapi plugin nova uses to read the logs.
In addition it ensures that any log that grows bigger
than 1MB will be truncated down to 5KB.
Ensuring the log directory is inside a loop back device
of a restricted size, and running this script every
minute using cron, will stop guests from using up
too much Dom0 disk space, and taking out the hypervisor
Part of blueprint xenapi-server-log
Change-Id: I4caa82f4d0620d924e37e3b605cf62b4d5b73570
Optimized grep calls to not create a grep process per file.
Defined variables to enhance readability.
Change-Id: Ic873f2c3b600a737ce54444dec973720c8813285
When there's 2 options with the same name, the script gets confused and
doesn't know in which group it goes. This fixes it, and fixes also the
template generation which is broken for various reasons:
- gettext.install was missing
- nova-rootwrap when imported calls exit() because it doesn't find a valid
configuration file, so let's exclude it entirely anyway
- eventlet/greendns needs to be ignored for this
Change-Id: Iaa4e9a806e79032ce1675b46a6b7a7628c3eff89
Signed-off-by: Julien Danjou <julien@danjou.info>
analyze_opts is a tool to help clean out a nova.conf file by checking
for unused and default values set in your nova.conf
Change-Id: I01fff134f8906bff929c55bea8c152653d3e9760
Mechanical translation of the deprecated
except x,y: construct with except x as y:
The latter works with any Python >= 2.6.
Add Hacking check.
Change-Id: I845829d97d379c1cd9b3a77e7e5786586f263b64
The dynamic nature of our object model is going to cause lots of
heartache for pylint, especially in the actual definitions. This
just excludes that whole directory from pylint's view, which will
limit the heartache to only everywhere objects are actually used.
Related to blueprint unified-object-model
Change-Id: I7f1a4b10b8a5098aab2ab2e8ea5da15ec43f0e2e
This failure in colorizier was hidding the actual error
in the tests. If we ran enough tests it also became difficult
to see which test had actually failed.
Fixes: bug 1183602
Change-Id: I806da8f13040b38258de7f3e0fa7b62700bdcead
Rename tools/pip-requires to requirements.txt and tools/test-requires
to test-requirements.txt. These are standard files, and tools in the
general world are growing intelligence about them.
Change-Id: I68ece8406fb1d4e082a42db8e76e17b1aaa7e775
Fixes: bug #1179008
Fixes bug 1182271.
* tools/pip-requires: The requests library released 1.2.1 today
which brings in sphinx as a new dependency. One of sphinx's
transitive dependencies is on a newer version of jinja than is
provided in the python-jinja2 package on CentOS 6, and since nova's
unit tests allow site-packages in the venv for libvirt support this
cap is needed so that gate-nova-python26 will continue to work.
Because requests is actually a dependency of python-cinderclient but
is only breaking nova's unit tests, it has to be added and come
before python-cinderclient in the pip-requires list. A separate
change has been proposed to openstack/requirements which will need
to merge first. https://review.openstack.org/29850
Change-Id: Ia198192d6231f71272d3aa4f5942828a45d86ceb
Remove a bunch of local custom scripts. Replace with configurable
external tools.
Use local hacking checks for nova specifics.
Change-Id: I75a01375ba0ec36d2ff05abc47abe0a3f225eda5
Sync some trivial Python 3 compatibility and pep8 improvements from oslo.
I just want to have this in sync so that pulling in a feature I'm working
on comes in without unrelated changes
Change-Id: I13fcb0bd538a153b90acf0b18497e335a64dc9d1
Declarative approach to packaging, obviates the need for copying files.
Also, removes the need for setup.py to import files from the nova
module.
./run_tests.sh now ensure's that nova.egg-info/entry_points.txt is up to
date before running tests.
Change-Id: I7b7a18e065a62491ade54fbed5d8400db1d4e6c7
Just changes openstack.common to use the new print statements.
This is in preparation for another sync coming shortly for the
scheduler library.
Change-Id: I17a86ef87f3253d6bb756cbd40a3ff5df9be23ff
These are all minor changes bringing nova up to commit
90e83530d4dc49d570fa05ea63a93805717dcfa0 in oslo-incubator.
Change-Id: I0291eed31b1e650da211fe2a8b65fad0c35c9053
Mark McLoughlin