Commit Graph

290 Commits

Author SHA1 Message Date
Sean Dague e239c5b712 remove flake8-import-order
This continues to make wrong guesses in many environments, and it only
slows down running pep8. Do not have this in the tree.

Change-Id: Ia63a279685160157385cc231c4b6dc2243f31d01
2017-04-07 15:06:06 -04:00
Jenkins bde8c7c615 Merge "tox: Stop calling config/policy generators twice" 2017-03-28 10:58:02 +00:00
Jenkins de0c41b5fe Merge "Tests: remove .testrepository/times.dbm in tox.ini (functional)" 2017-03-27 13:31:01 +00:00
Stephen Finucane 4e2441d2ff tox: Stop calling config/policy generators twice
oslo.config and oslo.policy provide Sphinx integrations for building and
showing sample config and policy files, respectively. These integrations
are registered as hooks that will fire when the builder is initialized
[1][2]. As a result, these commands do not need to be manually called.
Stop doing so.

[1] https://github.com/openstack/oslo.config/blob/3.23.0/oslo_config/sphinxconfiggen.py#L89
[2] https://github.com/openstack/oslo.policy/blob/1.21.0/oslo_policy/sphinxpolicygen.py#L73

Change-Id: I6c18b06c92370d28226f2814a3c094e60985cb83
2017-03-27 11:19:25 +01:00
Jenkins 4613b33674 Merge "Structure for simply managing placement-api-ref" 2017-03-24 20:17:52 +00:00
Chris Dent 8c59991363 Structure for simply managing placement-api-ref
This consists of a duplicate of the [nova-]api-ref setup and
conf.py along with tooling to fail the tox -edocs target when
a route that is defined in
nova.api.openstack.placement.handler.ROUTE_DECLARATIONS is not
present in placement-api-ref/source/index.rst.
tools/placement_api_docs.py will report which routes are missing.

Though completely gameable (as demonstrated in the current lame
index.rst) it's better than nothing and provides some useful
structuring on what to do next. It's also the case that the 'docs'
target in tox is not part of gating.

The response for GET / is in place with the necessary
parameters.yaml for it to be correctly described. The 'get-root.json'
file provides the JSON of the expected response. The expectation is
that later commits will add information for other urls and their
JSON files will be named method-path-separated-by-dash.json with a
request/response qualifier as necessary.

Followup patches will add other routes.

A new parameters.yaml is used instead of reusing the one from
api-ref as there isn't a lot of expected overlap and having a
separate file will ease eventual extraction.

Running tox -eplacement-api-ref will generate the docs for review,
with output in placement-api-ref/build/html/index.html.

This will be hooked up with CI to deploy the generated docs,
eventually.

Change-Id: Ifb4d91d39db0e49b55952e37cdfc9f63dcd37aa3
2017-03-24 15:15:53 +03:00
Matt Riedemann 8c6e132966 Tests: remove .testrepository/times.dbm in tox.ini (functional)
Change 98cbf72e57 handled this for
the py35 target but missed the functional-py35 target because
that one doesn't inherit the commands from the root testenv.

Change-Id: I018b809488f9e979f334c75415c15acb58dbd782
Related-Bug: #1229445
2017-03-21 16:06:41 -04:00
Jenkins 390c7e420f Merge "flake8: Specify 'nova' as name of app" 2017-03-21 18:11:06 +00:00
Jenkins f8939f76e9 Merge "remove flake8-import-order for test requirements" 2017-03-21 17:55:02 +00:00
Jenkins e3eac21382 Merge "More usage of ostestr and cleanup an unused dependency" 2017-03-20 16:04:47 +00:00
John L. Villalovos 07a042ba2c flake8: Specify 'nova' as name of app
Specify 'nova' as the name of the application for the flake8-import-order
plugin. That way it knows that imports of nova should come after external
libraries.

Fix issues discovered in ordering by this new check.

Change-Id: I822796ba3d750f93de813035aeee59e8ccb022a2
2017-03-16 13:48:51 -07:00
Sean Mooney a5d4adbd60 remove flake8-import-order for test requirements
- When flake8-import-order is installed the additional
  ordering tests it provides are enabled by default.
- When devstack installs python projects from source
  such as nova, the pip_install function installs both the
  requirements.txt and test-requiremets.txt dependencies systemwide.
  As a result, any python project that runs pep8 tests without a
  python virtual environment will also have import ordering enforced
  which may fail.
- This change removes flake8-import-order from the
  test-requirements.txt to prevent failures in projects other
  than nova that don't enforce pep8 import ordering.

Change-Id: I994df178f0000a0840f6658ccc2f277d5503f826
2017-03-16 17:36:54 +00:00
Matt Riedemann 2ce3b0d076 Introduce fast8 tox target
The pep8 target has the little known "-HEAD" argument
which makes it only run flake8 on the changes since the
last change in the history.

Even though that's nice, it's sometimes hard to remember
the -HEAD argument. Plus pep8 is running some other
checks on things like the docs.

So if all you want is the pep8 -- -HEAD behavior, this
is a simplied convenient form of it.

Change-Id: I97b277d8ac95ca3ef8379570e0e3886df26263e1
2017-03-15 13:00:10 -04:00
Stephen Finucane ac2a4abfca Duplicate JSON line ending check to pep8
The docs job is not run in the gate, meaning invalid JSON files with
\r\n line endings are able to sneak in. Duplicate the checks from the
docs environment to the pep8 environment so that gate jobs run it.

Note that pep8 is our general linting target and thus should include
these kind of tests.

Change-Id: Ie923902b04be79b9f66ba1841c60899a413f6439
Closes-Bug: #1673131
2017-03-15 16:05:38 +00:00
John L. Villalovos 66866b3ce6 Use flake8-import-order
Use the flake8 plugin flake8-import-order to check import ordering. It
can do it automatically and don't need reviewers to check it.

Change-Id: Ia3d81bbbb44b40804b3268c0e648276a36cb4805
2017-03-06 06:50:23 -08:00
Jenkins 1ef9cbdcf4 Merge "Tests: remove .testrepository/times.dbm in tox.ini" 2017-02-28 19:21:09 +00:00
Jenkins c990c73f4f Merge "Enable coverage report" 2017-02-28 13:54:26 +00:00
Maciej Szankin 98cbf72e57 Tests: remove .testrepository/times.dbm in tox.ini
Existance of this file causes py3* tests to fail if
run after py27. This patch adds cleaning up of this
file before running py3* tests.

Change-Id: I64ef1eac7f05f217674c73ddd2cc3af773dd75ce
Closes-Bug: #1229445
2017-02-27 18:17:51 +00:00
Luigi Toscano 8e34aece1a More usage of ostestr and cleanup an unused dependency
Use ostestr as runner instead of the custome pretty_tox.sh script
(pretty much removed everywhere). pretty_tox3.sh is just a wrapper
for ostestr but it could be probably removed as well going forward.
Also remove the tempest-lib dependency, which originally provided
subunit-trace (now from os-testr).

Change-Id: I4390964ce83167d634b80f696e82a0e199582ce6
2017-02-23 18:25:08 +01:00
Jenkins 0d0859573d Merge "Remove references to Python 3.4" 2017-02-23 12:43:41 +00:00
yushangbin 23b2adcc68 Enable coverage report
Enable coverage report in tox.ini.

Change-Id: Ifcb5d5d78e2a06e807050ce18e34e1d1f0fa1972
2017-02-21 19:51:47 +08:00
Gábor Antal 51c6b13847 Enable global hacking checks and removed local checks
In this patchset, I set up 'enabled-extensions' in flake8 section
of tox.ini. In newer hacking versions, we can enable some of the
non-default hacking rules (one by one), which are disabled by default
and implemented in the newer versions of hacking. The enabled extensions
are the following:

* [H106] Don’t put vim configuration in source files (off by default).
* [H203] Use assertIs(Not)None to check for None (off by default).
* [H904] Delay string interpolations at logging calls (off by default).

Together with enabling these rules, I also removed the locally
implemented versions of them. Due to limitations of local checking
engine, there were some places in the tests, where pep8 failed.

In this patchset, these codes are also fixed.

Change-Id: I3a9d2dc007a269cdb2cad441e22f5eb9b58ce0a0
2017-02-10 15:09:37 +01:00
Eric Brown 3b009a363e Remove references to Python 3.4
Now that there exists only a gate job for Python 3.5 and not 3.4,
we should remove those references to the 3.4 that is untested.

Change-Id: I4716c12a29427d2d101eb5527eb08096c147f333
2017-01-18 16:41:55 -08:00
Dina Belova ecc8de8d6c Integrate OSProfiler and Nova
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
  1) It checks that person who want to trace is trusted and knows
     secret HMAC key.
  2) It starts tracing in case of proper trace headers
     and adds the first wsgi trace point with info about the HTTP request

* Add initialization of osprofiler on start of a service
  Currently that includes oslo.messaging notifier instance creation
  to send Ceilometer backend notifications.

oslo-spec: https://review.openstack.org/#/c/103825/
python-novaclient change: https://review.openstack.org/#/c/254699/
based on: https://review.openstack.org/#/c/105096/

Co-Authored-By: Boris Pavlovic <boris@pavlovic.me>
Co-Authored-By: Munoz, Obed N <obed.n.munoz@intel.com>
Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com>
Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com>

Implements: blueprint osprofiler-support-in-nova
Change-Id: I82d2badc8c1fcec27c3fce7c3c20e0f3b76414f1
2017-01-18 15:00:14 +07:00
ChangBo Guo(gcb) 2738e25f7e Add Python 3.5 functional tests in tox.ini
Add Python 3.5 functional tests tox envlist, this is
needed by python 3.5 functional tests Jenkins job.

Change-Id: I1bba877fcc2c1f2140cbbba2bc6e6c20d7972615
2016-12-14 13:33:07 +08:00
Jenkins f60ef0dac7 Merge "Corrects the type of a base64 encoded string" 2016-12-13 23:03:13 +00:00
Jenkins f8d0f9ebf8 Merge "Add debug to tox environment" 2016-12-13 10:36:06 +00:00
EdLeafe a2d7ebd39b Corrects the type of a base64 encoded string
The nova/tests/functional/api_sample_tests/test_servers.py contains the
ServersSampleBase class, and in its class definition creates a
'user_data' attribute by base64 encoding a string. However, this will
not work in Python 3, as the base64.b64encode() method requires bytes,
not a string. As a result, importing the test class fails and no tests
get run.

Note that this change doesn't fix all tests to work under Python 3; it
simply fixes the bug that prevents the tests from running at all under
Python 3.

Closes-Bug: #1632856

Change-Id: I35a7b02132bed0387a173b339f6204bf0e3269de
2016-12-12 18:07:30 +00:00
Jenkins 9b0e0c8261 Merge "Check that all JSON files don't have \r\n in line" 2016-12-12 17:17:04 +00:00
liyingjun 9630b5105c Check that all JSON files don't have \r\n in line
The windows edited file may contain \r\n in line, add a check for
json files for the docs job.

Change-Id: I1ca30be139651ed49eb1f7867616dda6bd080884
2016-12-09 17:33:37 +08:00
Sergey Nikitin b8c88c69f5 Revert "reduce pep8 requirements to just hacking"
This reverts commit 1e287ae14b

Constraints mechanism doesn't work for 'hacking' lib:
I11eb0d762869ad8920795fb710f1b2eeb9354f12

We have to install all of test-requirements to get
'hacking' at the right version.

Closes-Bug: #1648054

Change-Id: I512192e8a9ab3a53a52e6cc4d4b8a222951af7fb
2016-12-07 16:48:52 +04:00
melanie witt b05e996ae5 Always use python2.7 for docs target
The docs testenv doesn't work with python 3.x on our codebase.
If someone is on a platform that defaults to python => python3,
building docs will fail for them.

Closes-Bug: #1646629

Change-Id: I56f3933e89e6fe9e181517325e2d7af5110c5656
2016-12-01 22:05:54 +00:00
Eric Brown e351cb698e Remove bandit.yaml in favor of defaults
With the latest version of bandit, the bandit.yaml is only
necessary for advanced tweaking.

Change-Id: I933a5389865fe577925ce3be30877ceae12fad31
2016-10-20 18:09:54 +00:00
Joshua Harlow b09d99d23e Explicitly name commands target environments
Naming envs like this is breaking the current
oslo periodic day job; a simple way to unbreak this
is to name them explicitly (which appears to be
more common).

So this changes the command listing to do just that
as well as lists py35 in the envlist (because its
a valid target).

Change-Id: I2a162604b60df8f18ff53069c6bf99cfecc4217e
2016-10-17 13:53:12 -07:00
Sivasathurappan Radhakrishnan df4cf3e5db Add debug to tox environment
The oslotest package distributes a shell file that may be used to assist
in debugging python code. The shell file uses testtools, and supports
debugging with pdb. Debug tox environment implements following test
instructions.
https://wiki.openstack.org/wiki/Testr#Debugging_.28pdb.29_Tests

To enable debugging, run tox with the debug environment. Below are the
following ways to run it.

 * tox -e debug module
 * tox -e debug module.test_class
 * tox -e debug module.test_class.test_method

Change-Id: I08937845803be7bd125b838ab07bda56f202e88d
Closes-Bug: 1632486
2016-10-13 15:52:20 +00:00
melanie witt 32e533d59a Always use python2.7 for functional tests
The functional testenv doesn't work with python 3.x on our codebase.
If someone is on a platform that defaults to python => python3,
functional tests will fail for them.

Closes-Bug: #1632521

Change-Id: I7bf6653f55c10d0a4f75054e519edf7da19c5c09
2016-10-12 00:02:50 +00:00
Jenkins 7a9eb10d0d Merge "Fix periodic-nova-py{27,35}-with-oslo-master" 2016-10-05 18:27:22 +00:00
Andreas Jaeger 951dee3e8e Fix periodic-nova-py{27,35}-with-oslo-master
The jobs periodic-nova-py27-with-oslo-master and
periodic-nova-py35-with-oslo-master are failing with:
"tox.ConfigError: ConfigError: substitution key '34,35' not found".

For details, see
http://logs.openstack.org/periodic/periodic-nova-py27-with-oslo-master/c01eab1/
http://logs.openstack.org/periodic/periodic-nova-py35-with-oslo-master/c10b034/

Fix this with not using a clever "py{34,35}" substitution in tox.ini and
instead duplicate the line.

Change-Id: If10fa53fff85350234ac1308505c273cde1333a1
2016-09-30 09:00:37 +02:00
Matt Riedemann 4f6f239efd Remove stale pyc files when running the cover job
The cover job runs the unit tests and having stale
pycs can skew those results, so remove the pyc files
before running the tests with --coverage.

Change-Id: I7393d2df36e715dbf53ba9ae6a077bdc8e79b5a5
2016-09-28 11:27:38 -04:00
Stephen Finucane 233160644f Revert "tox: Don't create '.pyc' files"
This reverts commit 0d6d1616b9.

The PYTHONDONTWRITEBYTECODE setting is ignored in post-v1.6.0 versions
of tox due to issues with setuptools [1]. Suffer the performance hit.

[1] https://tox.readthedocs.io/en/latest/changelog.html#id21

Change-Id: I99d165f0cc4f5382aef72f21f31bc985585f4748
2016-09-14 10:45:44 +01:00
melanie witt 43b36a4240 Pass GENERATE_HASHES to the tox test environment
When bumping nova object versions, it's necessary to generate new
hashes for those that have changed. New hashes are generated when the
GENERATE_HASHES environment variable is set. Let tox copy GENERATE_HASHES
to the test environment to allow object hash generation for a command
line like:

 GENERATE_HASHES=1 tox -epy27

Change-Id: Id652a7a9fbaeaf35db06744cbaf2484504fc2531
2016-09-08 23:50:25 +00:00
Jenkins ef5afcdc1d Merge "List system dependencies for running common tests" 2016-08-30 12:59:30 +00:00
Jenkins 3609487867 Merge "Add oslopolicy script runs to the docs tox target" 2016-08-25 10:37:38 +00:00
Andrew Laski 83bab97cbd Add oslopolicy script runs to the docs tox target
In an untouched codebase these two commands should not output anything
of interest as they are primarily tools for managing a custom
policy.json file. But it's useful to run them to ensure that they do not
error out.

Change-Id: I33e0b27164a0e9df240337c7e9b467f0f3859eac
2016-08-24 17:00:17 -04:00
Jeremy Stanley 7cdd5da7d7 List system dependencies for running common tests
Add a bindep.txt file containing a cross-platform list
of dependencies needed for running included tox-based tests. Also
include a tox environment for convenience calling the bindep[*]
utility to list any missing system requirements, and simplify the
contributor documentation to recommend this rather than embedding an
ad-hoc list of package names in an untestable document.

This change is self-testing.

[*] http://docs.openstack.org/infra/bindep/

Change-Id: I7c18b0921c39b99df89f2c356208303c2bad5a97
2016-08-24 06:49:32 +02:00
Jenkins 5e62bd3af6 Merge "Move JSON linting to pep8" 2016-08-18 15:57:28 +00:00
Takashi NATSUME d177395e75 Fix 'No data to report' error
Remove 2 unnecessary commands in 'testenv:cover' of tox.ini.
HTML coverage reports can be generated in 'cover' directory
by 'python setup.py testr --coverage' only
without the removed commands.

Change-Id: Ib9539f845aad29269a9cb07db719b22e35f7bbeb
Closes-Bug: #1613256
2016-08-17 02:52:17 +00:00
Andreas Jaeger af13871fc8 Use constraints for all tox environments
Infra is now ready to use constraints everywhere, adapt
the venv environment to use the default install command instead of a
specific non-constrainted install command.

With this change, now all tox environments use constraints.

Change-Id: Ibb73b4a27b5b0462a9e9f2dd80e437bcc8c7f8f4
2016-08-09 15:28:16 +02:00
Andreas Jaeger 190081c865 Move JSON linting to pep8
The docs job is not run in the gate, add the linting command from
the docs environment that checks for valid JSON files to the pep8
environement so that jobs run it.
Add a simple comment to both invocations to explain what it does.

Note that pep8 is our general linting target and thus should include
these kind of tests.

Change-Id: Ibb2ad29ca59b86fef41ebfe0ec3b8e8ea38e6d72
2016-08-09 13:07:09 +02:00
Andreas Jaeger 52f03d9324 Use constraints for releasenotes
OpenStack CI has now enabled constraints for releasenotes, use them -
remove the special install command.

Note that venv still cannot use constraints, this needs more changes
in OpenStack CI.

Change-Id: I125cb906f89c1de4227dc53521e1a60241ace997
Depends-On: If5d6b11bd795091eef949001cca6e8d4ec723ef2
2016-08-08 21:06:59 +02:00