Commit Graph

266 Commits

Author SHA1 Message Date
ChangBo Guo(gcb) 2738e25f7e Add Python 3.5 functional tests in tox.ini
Add Python 3.5 functional tests tox envlist, this is
needed by python 3.5 functional tests Jenkins job.

Change-Id: I1bba877fcc2c1f2140cbbba2bc6e6c20d7972615
2016-12-14 13:33:07 +08:00
Jenkins f60ef0dac7 Merge "Corrects the type of a base64 encoded string" 2016-12-13 23:03:13 +00:00
Jenkins f8d0f9ebf8 Merge "Add debug to tox environment" 2016-12-13 10:36:06 +00:00
EdLeafe a2d7ebd39b Corrects the type of a base64 encoded string
The nova/tests/functional/api_sample_tests/test_servers.py contains the
ServersSampleBase class, and in its class definition creates a
'user_data' attribute by base64 encoding a string. However, this will
not work in Python 3, as the base64.b64encode() method requires bytes,
not a string. As a result, importing the test class fails and no tests
get run.

Note that this change doesn't fix all tests to work under Python 3; it
simply fixes the bug that prevents the tests from running at all under
Python 3.

Closes-Bug: #1632856

Change-Id: I35a7b02132bed0387a173b339f6204bf0e3269de
2016-12-12 18:07:30 +00:00
Jenkins 9b0e0c8261 Merge "Check that all JSON files don't have \r\n in line" 2016-12-12 17:17:04 +00:00
liyingjun 9630b5105c Check that all JSON files don't have \r\n in line
The windows edited file may contain \r\n in line, add a check for
json files for the docs job.

Change-Id: I1ca30be139651ed49eb1f7867616dda6bd080884
2016-12-09 17:33:37 +08:00
Sergey Nikitin b8c88c69f5 Revert "reduce pep8 requirements to just hacking"
This reverts commit 1e287ae14b

Constraints mechanism doesn't work for 'hacking' lib:
I11eb0d762869ad8920795fb710f1b2eeb9354f12

We have to install all of test-requirements to get
'hacking' at the right version.

Closes-Bug: #1648054

Change-Id: I512192e8a9ab3a53a52e6cc4d4b8a222951af7fb
2016-12-07 16:48:52 +04:00
melanie witt b05e996ae5 Always use python2.7 for docs target
The docs testenv doesn't work with python 3.x on our codebase.
If someone is on a platform that defaults to python => python3,
building docs will fail for them.

Closes-Bug: #1646629

Change-Id: I56f3933e89e6fe9e181517325e2d7af5110c5656
2016-12-01 22:05:54 +00:00
Eric Brown e351cb698e Remove bandit.yaml in favor of defaults
With the latest version of bandit, the bandit.yaml is only
necessary for advanced tweaking.

Change-Id: I933a5389865fe577925ce3be30877ceae12fad31
2016-10-20 18:09:54 +00:00
Joshua Harlow b09d99d23e Explicitly name commands target environments
Naming envs like this is breaking the current
oslo periodic day job; a simple way to unbreak this
is to name them explicitly (which appears to be
more common).

So this changes the command listing to do just that
as well as lists py35 in the envlist (because its
a valid target).

Change-Id: I2a162604b60df8f18ff53069c6bf99cfecc4217e
2016-10-17 13:53:12 -07:00
Sivasathurappan Radhakrishnan df4cf3e5db Add debug to tox environment
The oslotest package distributes a shell file that may be used to assist
in debugging python code. The shell file uses testtools, and supports
debugging with pdb. Debug tox environment implements following test
instructions.
https://wiki.openstack.org/wiki/Testr#Debugging_.28pdb.29_Tests

To enable debugging, run tox with the debug environment. Below are the
following ways to run it.

 * tox -e debug module
 * tox -e debug module.test_class
 * tox -e debug module.test_class.test_method

Change-Id: I08937845803be7bd125b838ab07bda56f202e88d
Closes-Bug: 1632486
2016-10-13 15:52:20 +00:00
melanie witt 32e533d59a Always use python2.7 for functional tests
The functional testenv doesn't work with python 3.x on our codebase.
If someone is on a platform that defaults to python => python3,
functional tests will fail for them.

Closes-Bug: #1632521

Change-Id: I7bf6653f55c10d0a4f75054e519edf7da19c5c09
2016-10-12 00:02:50 +00:00
Jenkins 7a9eb10d0d Merge "Fix periodic-nova-py{27,35}-with-oslo-master" 2016-10-05 18:27:22 +00:00
Andreas Jaeger 951dee3e8e Fix periodic-nova-py{27,35}-with-oslo-master
The jobs periodic-nova-py27-with-oslo-master and
periodic-nova-py35-with-oslo-master are failing with:
"tox.ConfigError: ConfigError: substitution key '34,35' not found".

For details, see
http://logs.openstack.org/periodic/periodic-nova-py27-with-oslo-master/c01eab1/
http://logs.openstack.org/periodic/periodic-nova-py35-with-oslo-master/c10b034/

Fix this with not using a clever "py{34,35}" substitution in tox.ini and
instead duplicate the line.

Change-Id: If10fa53fff85350234ac1308505c273cde1333a1
2016-09-30 09:00:37 +02:00
Matt Riedemann 4f6f239efd Remove stale pyc files when running the cover job
The cover job runs the unit tests and having stale
pycs can skew those results, so remove the pyc files
before running the tests with --coverage.

Change-Id: I7393d2df36e715dbf53ba9ae6a077bdc8e79b5a5
2016-09-28 11:27:38 -04:00
Stephen Finucane 233160644f Revert "tox: Don't create '.pyc' files"
This reverts commit 0d6d1616b9.

The PYTHONDONTWRITEBYTECODE setting is ignored in post-v1.6.0 versions
of tox due to issues with setuptools [1]. Suffer the performance hit.

[1] https://tox.readthedocs.io/en/latest/changelog.html#id21

Change-Id: I99d165f0cc4f5382aef72f21f31bc985585f4748
2016-09-14 10:45:44 +01:00
melanie witt 43b36a4240 Pass GENERATE_HASHES to the tox test environment
When bumping nova object versions, it's necessary to generate new
hashes for those that have changed. New hashes are generated when the
GENERATE_HASHES environment variable is set. Let tox copy GENERATE_HASHES
to the test environment to allow object hash generation for a command
line like:

 GENERATE_HASHES=1 tox -epy27

Change-Id: Id652a7a9fbaeaf35db06744cbaf2484504fc2531
2016-09-08 23:50:25 +00:00
Jenkins ef5afcdc1d Merge "List system dependencies for running common tests" 2016-08-30 12:59:30 +00:00
Jenkins 3609487867 Merge "Add oslopolicy script runs to the docs tox target" 2016-08-25 10:37:38 +00:00
Andrew Laski 83bab97cbd Add oslopolicy script runs to the docs tox target
In an untouched codebase these two commands should not output anything
of interest as they are primarily tools for managing a custom
policy.json file. But it's useful to run them to ensure that they do not
error out.

Change-Id: I33e0b27164a0e9df240337c7e9b467f0f3859eac
2016-08-24 17:00:17 -04:00
Jeremy Stanley 7cdd5da7d7 List system dependencies for running common tests
Add a bindep.txt file containing a cross-platform list
of dependencies needed for running included tox-based tests. Also
include a tox environment for convenience calling the bindep[*]
utility to list any missing system requirements, and simplify the
contributor documentation to recommend this rather than embedding an
ad-hoc list of package names in an untestable document.

This change is self-testing.

[*] http://docs.openstack.org/infra/bindep/

Change-Id: I7c18b0921c39b99df89f2c356208303c2bad5a97
2016-08-24 06:49:32 +02:00
Jenkins 5e62bd3af6 Merge "Move JSON linting to pep8" 2016-08-18 15:57:28 +00:00
Takashi NATSUME d177395e75 Fix 'No data to report' error
Remove 2 unnecessary commands in 'testenv:cover' of tox.ini.
HTML coverage reports can be generated in 'cover' directory
by 'python setup.py testr --coverage' only
without the removed commands.

Change-Id: Ib9539f845aad29269a9cb07db719b22e35f7bbeb
Closes-Bug: #1613256
2016-08-17 02:52:17 +00:00
Andreas Jaeger af13871fc8 Use constraints for all tox environments
Infra is now ready to use constraints everywhere, adapt
the venv environment to use the default install command instead of a
specific non-constrainted install command.

With this change, now all tox environments use constraints.

Change-Id: Ibb73b4a27b5b0462a9e9f2dd80e437bcc8c7f8f4
2016-08-09 15:28:16 +02:00
Andreas Jaeger 190081c865 Move JSON linting to pep8
The docs job is not run in the gate, add the linting command from
the docs environment that checks for valid JSON files to the pep8
environement so that jobs run it.
Add a simple comment to both invocations to explain what it does.

Note that pep8 is our general linting target and thus should include
these kind of tests.

Change-Id: Ibb2ad29ca59b86fef41ebfe0ec3b8e8ea38e6d72
2016-08-09 13:07:09 +02:00
Andreas Jaeger 52f03d9324 Use constraints for releasenotes
OpenStack CI has now enabled constraints for releasenotes, use them -
remove the special install command.

Note that venv still cannot use constraints, this needs more changes
in OpenStack CI.

Change-Id: I125cb906f89c1de4227dc53521e1a60241ace997
Depends-On: If5d6b11bd795091eef949001cca6e8d4ec723ef2
2016-08-08 21:06:59 +02:00
Sean Dague 98ad916659 update wording around pep8 exceptions
At least once a cycle we get contributions from new folks that attempt
to update a bunch of pep8 formatting. They believe that this is in
good faith helping, but don't realize we really aren't likely to
accept that.

Update the text here to explain that more clearly, so that people
don't keep falling down this rabbit hole.

Change-Id: Iec383a18cb3d64b9655e5bd1d82cabd3e785686c
2016-08-04 15:51:12 +00:00
Jenkins b5822aa446 Merge "Reminder that release notes are built from commits" 2016-08-03 10:44:04 +00:00
Artom Lifshitz 5ee95b3240 Reminder that release notes are built from commits
This patch introduces a new tools/releasenotes_tox.sh script for the
tox -e releasenotes test environment. It will detect when there are
uncommitted release notes and print a reminder to commit them.

Change-Id: I54af12d7eb20b37263e23443900745884cffacea
2016-07-29 13:32:33 -04:00
Jenkins b6a0066855 Merge "Update tox.ini: Constraints are possible for api* jobs" 2016-07-28 01:23:10 +00:00
Jenkins 6cf0e9e213 Merge "tox: Use conditional targets" 2016-07-25 18:38:03 +00:00
Jenkins aaeaaf5c16 Merge "tox: Don't create '.pyc' files" 2016-07-25 18:37:10 +00:00
Andreas Jaeger 064c4db632 Update tox.ini: Constraints are possible for api* jobs
OpenStack CI setup has been changed so that api jobs can
use constraints, remove the extra install_command.

Change-Id: I1eb8c5b74adf28e8b6267f25b116341c540729dd
Depends-On: I8f45a53429b9fcbf3689a268f096afdf5f32f461
2016-07-21 10:27:28 +02:00
Andreas Jaeger 1a95ac48b7 Use constraints for coverage job
OpenStack CI supports now constraints in the coverage job - as a first
job to check that recent changes for zuul-cloner work correctly.

Use constraints for cover environment. If this works successfully, the
other environments can be updated in followups.

Change-Id: I6069bee0a89d38f6a3ec57418e7bda9850dc6f64
2016-07-17 15:50:02 +02:00
Stephen Finucane 17630d37fc tox: Use conditional targets
This simplifies our tox file and reduces duplication.

Change-Id: I085899e00a249fb850c81737b1088bb82ce820ba
2016-07-11 17:50:26 +01:00
Stephen Finucane 0d6d1616b9 tox: Don't create '.pyc' files
There's no reason to create these files and cleaning them up takes
precious time during each test run. Don't do it.

The benefits of this can be seen in running both the full test suite
and part of the test suite. For the former case - running the full
test suite:

    $ time ( tox -e py27 )

Without this change:

    real    12m48.003s
    user    38m27.039s
    sys     1m2.786s

With this change:

    real    11m53.921s
    user    39m33.360s
    sys     1m1.686s

For the latter case - running part of the full test suite:

    $ time ( tox -e py27 -- nova.tests.unit.test_configdrive2 )

Without this change:

    real    2m0.686s
    user    1m3.080s
    sys     0m12.950s

With this change:

    real    1m37.098s
    user    1m16.559s
    sys     0m11.846s

The actual results can vary massively, but this change is always
faster.

Change-Id: Iffd759e368878f39c015258935d8b37de09d9745
2016-07-11 17:48:32 +01:00
Jens Rosenboom df0aa8acaa Add a py35 environment to tox
Infra will start running python35 jobs soon [1], targeting to replace
the current python34 jobs. This is due to the change from Ubuntu Trusty
to Xenial, where only python3.5 will be available.

So we add a py35 environment with the same definitions for skipping
not-yet-working checks as we have in place for py34 already.

[1] https://review.openstack.org/336272

Change-Id: I017478fafa31121f40c95d73e49ac4dced80c7a2
2016-07-01 17:04:41 +02:00
Jenkins d68acca919 Merge "Add ability to select specific tests for py34" 2016-06-29 16:31:23 +00:00
Andrew Laski 5e38fa3cb5 Add policy sample generation
This adds the entry point and config file necessary for using the
oslo.policy sample generation script. It also adds a tox target to
simplify the usage of it.

The command is also run in the docs tox target since the sample file
can be considered documentation.

Change-Id: I18a1fa567fc04b0e0a37c02f20024b2ce483f060
Partially-Implements: bp policy-in-code
2016-06-28 20:52:44 +00:00
Jens Rosenboom 0196f4676e Add ability to select specific tests for py34
For py27 you can simply run specific tests by calling

    tox -e py27 nova.tests.unit.test_something

Add a similiar option for the py34 environment. This will skip the
blacklist of tests that would otherwise be ignored for python3.

Change-Id: I9bf5901afed4abf592e1aaff2f4820b9e3abc671
2016-06-28 05:24:48 +02:00
Akihiro Motoki c30562f458 tox.ini: Remove unnecessary comments in api-ref target
We pass -W option to sphinx-build to generate api-ref now.
The comment looks no longer correct.

Change-Id: Ie5dfaf46e9d3369c32c43da07c02501b224fb092
2016-06-10 04:23:52 +00:00
Jenkins f48c58915d Merge "Let setup.py compile_catalog process all language files" 2016-05-24 20:56:24 +00:00
melanie witt c19dd499f0 Pass OS_DEBUG to the tox test environment
The StandardLogging fixture defaults to log level INFO and has
the ability to log at level DEBUG if the OS_DEBUG environment
variable is set to a true value. Let tox copy OS_DEBUG to the
test environment to allow debug log capture for a command line
like:

  OS_DEBUG=True tox -epy27

Change-Id: Ibe51653f652c99aee56f702be571130cbb7f913d
2016-05-21 01:48:06 +00:00
Sven Anderson ece7b8f1e7 Let setup.py compile_catalog process all language files
Two years ago the translation files have been split into several
files, separating the log messages of different log levels from each
other, like X.pot, X-log-warning.pot, X-log-info.pot, and so on.
However, the setup.py command `compile_catalogs`, that comes from the
babel package and compiles the corresponding .po files into .mo
files, only supported one file per python package.  This means that
during packaging `compile_catalogs` never compiled the X-log-*.po
files, so the corresponding translations were always missing.

Since babel 2.3 the domain can be set to a space separated list of
domains.  This change adds the the additional log level files to the
domain list.

The obsolete check that .po and .pot files are valid is removed from
tox.ini.

Change-Id: I0d9cb16ac50caae570c57a1db6bed656bf13c3dc
Closes-Bug: #1536226
2016-05-20 17:56:06 +02:00
Sean Dague 1555736e3c Final warnings removals for api-ref
This removes the final warnings from the api-ref code base, and flips
the sphinx switch to enforce warnings as errors.

It also adds code to the rest_parameters extension to make it more
clear where a problem is when it comes to included parameters. This
puts us in a place where we can start doing the file per file look at
content.

Part of bp:api-ref-in-rst

Change-Id: Ic2c99d96d6addcafa00b9f16785c2fe59b1798d3
2016-04-20 11:25:30 -04:00
ChangBo Guo(gcb) b1871e5759 Nuke cliutils from oslo-incubator
Oslo team won't create new libary to include functions of module cliutils.
Nova only use two methods in cmd/manage.py, so copy these two methods
to nova/utils.py and clean up oslo-incubator related stuff.

Note:There is tiny change of method validate_args to make it Python 3
compatible.

Change-Id: Ibef5076f9556fa3224c2ca32e8c7fe2c4c1f739b
2016-04-03 23:38:31 +08:00
Sean Dague 329898ffbc Add sample API content
This adds sample API content for versions and servers resources,
including the parameters.yaml that is needed in both of these cases.

It also makes a new tox.ini target 'api-ref' for building these docs,
which will be used as part of the publishing pipeline.

Change-Id: I310ed352dc5dd81d01f2fd5f1a2fab662c29f0dc
2016-03-29 16:35:33 -04:00
Andreas Jaeger f25648a963 Error on API Guide warnings
Error on warnings in building the API guide and fix any problem
encountered:
* tox.ini: Enable tread warnings as errors
* index.rst: Add unreferenced file extra_specs_and_properties as
  hidden file so that it does not show up in index. It is still
  built and will be referenced from other files.
* general_info.rst: Remove reference to non-existing file
* server_concept: Add missing empty line after code directives

Also fix a typo and remove extra EOL whitespace.

Change-Id: Ie766f8d2f7d82b7b5b02541b342f9829ca4550a0
2016-03-17 07:57:23 +01:00
Andreas Jaeger 7c7a6e0d1b Do not use constraints for venv
The venv environment is used by infra scripts to build tarballs and
documents - and used by both check and post jobs. In post jobs, we
currently do not support constraints, so add the non-constraints install
command here so that nova-docs (publishing of documents to
docs.openstack.org) and the tarball jobs work again.

Change-Id: I2e5c0a4d739296b8b9d4ef7f58b69e1eff3c9575
2016-02-28 19:06:50 +01:00
Sean Dague 1e287ae14b reduce pep8 requirements to just hacking
Because we are constrained by the constraints mechanism we no longer
need to install all of test-requirements to get hacking at the right
version, because we'll only install the upper-constraints version.

Change-Id: I877ffec045cc43a32f97be8afe453d4755507217
2016-02-24 20:23:05 -05:00