OpenStack 单元测试

OpenStack 单元测试

OpenStack开发——单元测试

      本文将介绍OpenStack单元测试的部分。本文将重点讲述Python和OpenStack中的单元测试的生态环境。

      openstack社区推崇的是使用tox进行单元测试，tox只需要配置好tox.ini就可以了，比较灵活也比较简单。在opensatck的项目代码中也有包含tox配置，安装好tox之后就可以对项目代码进行单元测试

通过demo学习OpenStack开发——单元测试
  单元测试的重要性
  单元测试工具
     unittest
     mock
     testtools
     fixtures
     testscenarios
     subunit
     testrepository
     coverage
     tox
     单元测试工具小结
  Keystone的单元测试框架
     使用tox进行测试环境管理
     使用testrepository管理测试的运行
  单元测试用例的代码架构
  总结
  系列后记

单元测试工具

    Python的单元测试工具很多，为单元测试提供不同方面的功能。OpenStack的项目也基本把现在流行的单元测试工具都用全了。单元测试可以说是入门OpenStack开发的最难的部分，也是最后一公里。本文，我们就介绍一下在OpenStack中会用到的单元测试的工具。由于数量很多，不可能详细介绍，因此主要做一些概念和用途上的介绍。

unittest

          unittest是Python的标准库，提供了最基本的单元测试功能，包括单元测试运行器（简称runner）和单元测试框架。项目的单元测试代码的测试类可以继承unittest.TestCase类，这样这个类就能够被runner发现并且执行。同时，unittest.TestCase这个类还定义了setUp()，tearDown()，setUpClass()和tearDownClass()方法，是用来运行单元测试前的设置工作代码和单元测试后的清理工作代码，这个也是所有Python代码遵守的规范，所以第三方的单元测试库和框架也都遵循这个规范。unittest库也提供了一个runner，可以使用$ python -m unittest test_module的命令来执行某个模块的单元测试。另外，在Python中指定要运行的单元测试用例的完整语法是：path.to.your.module:ClassOfYourTest.test_method。unittest是学习Python单元测试最基本也最重要的一个库，完整的说明请查看查看官方文档。
。

mock

             mock也是另一个重要的单元测试库，在Python 2中是作为一个第三方库被使用的，到Python 3时，就被纳入了标准库，可见这个库的重要性。简单的说，mock就是用来模拟对象的行为，这样在进行单元测试的时候，可以指定任何对象的返回值，便于测试对外部接口有依赖的代码。关于mock的使用，可以查看我之前写的这篇文章Python Mock的入门。

testtools

             testtools是个unittest的扩展框架，主要是在unittest的基础上提供了更好的assert功能，使得写单元测试更加方便。具体可以查看文档。
。

fixtures

         fixture的意思是固定装置，在Python的单元测试中，是指某段可以复用的单元测试setUp和tearDown代码组合。一个fixture一般用来实现某个组件的setUp和tearDown逻辑，比如测试前要先创建好某些数据，测试后要删掉这些数据，这些操作就可以封装到一个fixture中。这样不同的测试用例就不用重复写这些代码，只要使用fixture即可。fixtures模块是一个第三方模块，提供了一种简单的创建fixture类和对象的机制，并且也提供了一些内置的fixture。具体的使用方法可以查看官方文档。
。

testscenarios

         testscenarios模块满足了场景测试的需求。它的基本用法是在测试类中添加一个类属性scenarios，该属性是一个元组，定义了每一种场景下不同的变量的值。比如说你测试一段数据访问代码，你需要测试该代码在使用不同的驱动时，比如MongoDB、SQL、File，是否都能正常工作。我们有三种办法：

最笨的办法是为不同的驱动把同一个测试用例编写3遍。
 比较好的办法是，编写一个统一的非测试用例方法，接收driver作为参数，执行测试逻辑，然后再分别编写三个测试用例方法去调用这个非测试用例方法。
 更好的办法就是使用testscenarios模块，定义好scenarios变量，然后实现一个测试用例方法。
 testscenarios模块在OpenStack Ceilometer中被大量使用。更多的信息可以查看文档。

subunit

          subunit是一个用于传输单元测试结果的流协议。一般来说，运行单元测试的时候是把单元测试的结果直接输出到标准输出，但是如果运行大量的测试用例，这些测试结果就很难被分析。因此就可以使用python-subunit模块来运行测试用例，并且把测试用例通过subunit协议输出，这样测试结果就可以被分析工具聚合以及分析。python-subunit模块自带了一些工具用来解析subunit协议，比如你可以这样运行测试用例：$ python -m subunit.run test_module | subunit2pyunit，subunit2pyunit命令会解析subunit协议，并且输出到标准输出。关于subunit的更多信息，请查看官方文档。

testrepository

          OpenStack中使用testrepository模块管理单元测试用例。当一个项目中的测试用例很多时，如何更有效的处理单元测试用例的结果就变得很重要。testrepository的出现就是为了解决这个问题。testrepository使用python-subunit模块来运行测试用例，然后分析subunit的输出并对测试结果进行记录（记录到本地文件）。举例来说，testrepository允许你做这样的事情：

知道哪些用例运行时间最长
 显示运行失败的用例
 重新运行上次运行失败的用例
 testrepository的更多信息，请查看官方文档。

coverage

           coverage是用来计算代码运行时的覆盖率的，也就是统计多少代码被执行了。它可以和testrepository一起使用，用来统计单元测试的覆盖率，在运行完单元测试之后，输出覆盖率报告。具体的使用方法可以查看官方文档。

tox

           tox是用来管理和构建虚拟环境(virtualenv)的。对于一个项目，我们需要运行Python 2.7的单元测试，也需要运行Python 3.4的单元测试，还需要运行PEP8的代码检查。这些不同的任务需要依赖不同的库，所以需要使用不同的虚拟环境。使用tox的时候，我们会在tox的配置文件tox.ini中指定不同任务的虚拟环境名称，该任务在虚拟环境中需要安装哪些包，以及该任务执行的时候需要运行哪些命令。更多信息，请查看官方文档。

单元测试工具小结

     本文介绍了OpenStack中常用的单元测试工具的基本用途，希望大家对这些工具有个大概的认识。这里我们可以按照类别总结一下这些工具：

  • 测试环境管理: tox

          使用tox来管理测试运行的虚拟环境，并且调用testrepository来执行测试用例。

  • 测试用例的运行和管理: testrepository, subunit, coverage

          testrepository调用subunit来执行测试用例，对测试结果进行聚合和管理；调用coverage来执行代码覆盖率的计算。

  • 测试用例的编写: unittest, mock, testtools, fixtures, testscenarios

          使用testtools作为所有测试用例的基类，同时应用mock, fixtures, testscenarios来更好的编写测试用例。

     接下来我们来分析Keystone项目的单元测试框架，可以让你看到在OpenStack的实际项目中，这些工具是如何被使用的。

Keystone的单元测试框架

      现在，我们以Keystone项目为例，来看下真实项目中的单元测试是如何架构的。我们采用自顶向下的方式，先从最上层的部分介绍起。

安装tox (Centos 7)

1.安装pip

easy_install pip

2.设置pip源

pip官方源下载比较慢，我们可以设置一个国内的源。

$ mkdir ~/.pip

$ vim~/.pip/pip.conf

[global]

timeout =6000

index-url= http://mirrors.aliyun.com/pypi/simple

[install]

trusted-host= mirrors.aliyun.com

3.安装tox

pip install tox

3.安装依赖插件

yum install gcc libffi-devel python-devel openssl-devel
yum install git

yum install mysql-devel
yum install postgresql postgresql-devel python-devel
yum install libxslt-devel libxml2-devel

使用tox进行测试环境管理

大部分情况下，我们都是通过tox命令来执行单元测试的，并且传递环境名称给tox命令：

[root@qqzhangl keystone]# tox -e py27

tox命令首先会读取项目根目录下的tox.ini文件，获取相关的信息，然后根据配置构建virtualenv，保存在.tox/目录下，以环境名称命名：

[root@qqzhangl keystone] ls .tox log pep8 py27

除了log目录，其他的都是普通的virtualenv环境，你可以自己查看一下内容。我们来看下py27这个环境的相关配置（在tox.ini）中，我直接在内容上注释一些配置的用途：

[tox]

minversion = 1.6

skipsdist = Trueenvlist = py34,py27,pep8,docs,genconfig,releasenotes 

[testenv] # testenv是默认配置，如果某个配置在环境专属的section中没有，就从这个section中读取

usedevelop = True # usedevelop表示安装virtualenv的时候，本项目自己的代码采用开发模式安装，

也就是不会拷贝代码到virtualenv目录中，只是做个链接

install_command = pip install -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?h=stable/mitaka} {opts} {packages} # install_command表示构建环境的时候要执行的命令，一般是使用pip安装

setenv = VIRTUAL_ENV={envdir}  

deps = -r{toxinidir}/test-requirements.txt

       .[ldap,memcache,mongodb]  # deps指定构建环境的时候需要安装的依赖包，这个就是作为pip命令的参数 # keystone这里使用的写法比较特殊一点，第二行的.[ldap,memcache,mongodb]是两个依赖，

第一个点'.'表示当前项目的依赖，也就是requirements.txt，第二个部分[ldap,memcache,mongodb]

表示extra，是在setup.cfg文件中定义的一个段的名称，该段下定义了额外的依赖，这些可以查看PEP0508# 一般的项目这里会采用更简单的方式来书写，直接安装两个文件中的依赖：

#    -r{toxinidir}/requirements.txt

#    -r{toxinidir}/test-requirements.txt

commands =

  find keystone -type f -name "*.pyc" -delete

  bash tools/pretty_tox.sh '{posargs}' # commands表示构建好virtualenv之后要执行的命令，这里调用了tools/pretty_tox.sh来执行测试

whitelist_externals =

  bash

  find

passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY PBR_VERSION

[testenv:py34]   # 这个section是为py34环境定制某些配置的，没有定制的配置，从[testenv]读取

commands =

  find keystone -type f -name "*.pyc" -delete

  bash tools/pretty_tox_py3.sh

[testenv:legacy_drivers]

deps = -r{toxinidir}/test-requirements.txt

       nose

       .[ldap,memcache,mongodb]

commands =

  # Run each legacy test separately, to avoid SQL model redefinitions

  find keystone -type f -name "*.pyc" -delete

  nosetests -v 

      keystone/tests/unit/backend/legacy_drivers/assignment/V8/sql.py

  nosetests -v 

      keystone/tests/unit/backend/legacy_drivers/role/V8/sql.py

  nosetests -v 

      keystone/tests/unit/backend/legacy_drivers/federation/V8/api_v3.py

  nosetests -v 

      keystone/tests/unit/backend/legacy_drivers/resource/V8/sql.py

[testenv:pep8] #代码编码规范

deps =

     .[bandit]

     {[testenv]deps}

commands =

  flake8

  # Run bash8 during pep8 runs to ensure violations are caught by

  # the check and gate queues

  bashate examples/pki/gen_pki.sh

  # Check that .po and .pot files are valid.

  bash -c "find keystone -type f -regex '.*.pot?' -print0| 

           xargs -0 -n 1 msgfmt --check-format -o /dev/null"

  # Run security linter

  bandit -r keystone -x tests

[testenv:bandit]

# NOTE(browne): This is required for the integration test job of the bandit

# project. Please do not remove.

deps = .[bandit]

commands = bandit -r keystone -x tests

[testenv:cover] #代码覆盖率

# Also do not run test_coverage_ext tests while gathering coverage as those

# tests conflict with coverage.

# NOTE(sdague): this target does not use constraints because

# upstream infra does not yet support it. Once that's fixed, we can

# drop the install_command.

install_command = pip install -U --force-reinstall {opts} {packages}

commands =

  find keystone -type f -name "*.pyc" -delete

  python setup.py testr --coverage --testr-args='{posargs}'

[testenv:venv]

# NOTE(jaegerandi): this target does not use constraints because

# upstream infra does not yet support it. Once that's fixed, we can

# drop the install_command.

install_command = pip install -U --force-reinstall {opts} {packages}

commands = {posargs}

[testenv:debug] #代码调试

commands =

  find keystone -type f -name "*.pyc" -delete

  oslo_debug_helper {posargs}

passenv =

    KSTEST_ADMIN_URL

    KSTEST_ADMIN_USERNAME

    KSTEST_ADMIN_PASSWORD

    KSTEST_ADMIN_DOMAIN_ID

    KSTEST_PUBLIC_URL

    KSTEST_USER_USERNAME

    KSTEST_USER_PASSWORD

    KSTEST_USER_DOMAIN_ID

    KSTEST_PROJECT_ID

[testenv:functional] 

basepython = python3.4

deps = -r{toxinidir}/test-requirements.txt

setenv = OS_TEST_PATH=./keystone/tests/functional

commands =

  find keystone -type f -name "*.pyc" -delete

  python setup.py testr --slowest --testr-args='{posargs}'

passenv =

    KSTEST_ADMIN_URL

    KSTEST_ADMIN_USERNAME

    KSTEST_ADMIN_PASSWORD

    KSTEST_ADMIN_DOMAIN_ID

    KSTEST_PUBLIC_URL

    KSTEST_USER_USERNAME

    KSTEST_USER_PASSWORD

    KSTEST_USER_DOMAIN_ID

    KSTEST_PROJECT_ID

[flake8]

filename= *.py,keystone-all,keystone-manage

show-source = true

# D100: Missing docstring in public module

# D101: Missing docstring in public class

# D102: Missing docstring in public method

# D103: Missing docstring in public function

# D104: Missing docstring in public package

# D105: Missing docstring in magic method

# D202: No blank lines allowed after docstring.

# D203: 1 blank required before class docstring.

# D205: Blank line required between one-line summary and description.

# D400: First line should end with a period.

# D401: First line should be in imperative mood.

ignore = D100,D101,D102,D103,D104,D105,D203,D205,D400,D401

exclude=.venv,.git,.tox,build,dist,doc,*openstack/common*,*lib/python*,*egg,tools,vendor,.update-venv,*.ini,*.po,*.pot

max-complexity=24

[testenv:docs] #生成代码文档

commands=

    bash -c "rm -rf doc/build"

    bash -c "rm -rf doc/source/api"

    python setup.py build_sphinx

[testenv:releasenotes] #生成版本信息

# NOTE(sdague): this target does not use constraints because

# upstream infra does not yet support it. Once that's fixed, we can

# drop the install_command.

install_command = pip install -U --force-reinstall {opts} {packages}

commands = sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html

[testenv:genconfig]  #生成配置文件

commands = oslo-config-generator --config-file=config-generator/keystone.conf

[hacking]

import_exceptions =

  keystone.i18n

  six.moves

local-check-factory = keystone.tests.hacking.checks.factory

  setup.cfg的extra部分如下：

 [extras]
ldap =
  python-ldap>=2.4:python_version=='2.7' # PSF
  ldappool>=1.0:python_version=='2.7' # MPL
memcache =
  python-memcached>=1.56 # PSF
mongodb =
  pymongo!=3.1,>=3.0.2 # Apache-2.0
bandit =
  bandit>=0.17.3 # Apache-2.0

使用testrepository管理测试的运行

上面我们看到tox.ini文件中的commands参数中执行的是_tools/pretty_tox.sh_命令。这个脚本的内容如下：

#!/usr/bin/env bash

set -o pipefail

TESTRARGS=$1

# testr和setuptools已经集成，所以可以通过setup.py testr命令来执行
# --testr-args表示传递给testr命令的参数，告诉testr要传递给subunit的参数
# subunit-trace是os-testr包中的命令（os-testr是OpenStack的一个项目），用来解析subunit的输出的。
python setup.py testr --testr-args="--subunit $TESTRARGS" | subunit-trace -f   
retval=$?
# NOTE(mtreinish) The pipe above would eat the slowest display from pbr's testr
# wrapper so just manually print the slowest tests.
echo -e "
Slowest Tests:
"

# 测试结束后，让testr显示出执行时间最长的那些测试用例
testr slowest
exit $retval

tox就是从tools/pretty_tox.sh这个命令开始调用testr来执行单元测试的。testr本身的配置是放在项目根目录下的.testr.conf文件：

[DEFAULT]
test_command=
    ${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./keystone/tests/unit} $LISTOPT $IDOPTION

test_id_option=--load-list $IDFILE
test_list_option=--list
group_regex=.*(test_cert_setup)


# NOTE(morganfainberg): If single-worker mode is wanted (e.g. for live tests)
# the environment variable ``TEST_RUN_CONCURRENCY`` should be set to ``1``. If
# a non-default (1 worker per available core) concurrency is desired, set
# environment variable ``TEST_RUN_CONCURRENCY`` to the desired number of
# workers.
test_run_concurrency=echo ${TEST_RUN_CONCURRENCY:-0}

这个文件中的配置项可以从testr官方文档中找到。其中test_command命令表示要执行什么命令来运行测试用例，这里使用的是subunit.run，这个我们在上面提到过了。

到目前为止的流程就是：

  1. tox建好virtualenv

  2. tox调用testr

  3. testr调用subunit来执行测试用例

每个OpenStack项目基本上也都是这样。如果你自己在开发一个Python项目，你也可以参考这个架构。

 

单元测试用例的代码架构

        下面我们来看一下Keystone的单元测试代码是如何写的，主要是看一下其层次结构。每个OpenStack项目的单元测试代码结构可能都不一样，不过你了解完Keystone的结构之后，看其他项目的就会比较快了。

我们以一个测试类为例来分析测试代码的结构：keystone.tests.unit.test_v3_assignment:AssignmentTestCase。下面是这个类的继承结构，同一级别的缩进表示多重继承，增加缩进表示父类，这里删掉了不必要的路径前缀（从unit目录开始，如下所示：

# 这个测试类是测RoleAssignment的API的
unit.test_v3_assignment.RoleAssignmentBaseTestCase
-> unit.test_v3.AssignmentTestMixin  这个类包含了一下测试Assignment的工具函数
-> unit.test_v3.RestfulTestCase      这个类是进行V3 REST API测试的基类，实现了V3 API的请求发起和校验
  -> unit.core.SQLDriverOverride     用于修改各个配置的driver字段为sql
  -> unit.test_v3.AuthTestMixin      包含创建认证请求的辅助函数
  -> unit.rest.RestfulTestCase       这个类是进行RESP API测试的基类，V2和V3的API测试
都是以这个类为基类，这个类的setUp方法会初始化数据库，创建好TestApp。
    -> unit.TestCase                 这个类是Keystone中所有单元测试类的基类，它主要初始化配置，以及初始化log
      -> unit.BaseTestCase           这个类主要是配置测试运行的基本环境，修改一些环境变量，比如HOME等。
          -> testtools.TestCase      使用testtools作为测试框架
            -> unittest.TestCase     testtools本身是unittest的扩展
[root@qqzhangl keystone]# tox -e py27 keystone.tests.unit.test_v3_assignment.AssignmentTestCase
 py27 develop-inst-noop: /home/unittest/keystone
 py27 installed: amqp==1.4.9,anyjson==0.3.3,appdirs==1.4.0,Babel==2.2.0,bashate==0.4.0,beautifulsoup4==4.4.1,cachetools==1.1.5,cffi==1.5.2,contextlib2==0.5.1,coverage==4.0.3,cryptography==1.2.3,debtcollector==1.3.0,decorator==4.0.9,docutils==0.12,dogpile.cache==0.5.7,dogpile.core==0.4.1,dulwich==0.18.3,ecdsa==0.13,enum34==1.1.2,eventlet==0.18.4,extras==0.0.3,fasteners==0.14.1,fixtures==1.4.0,flake8==2.2.4,flake8-docstrings==0.2.1.post1,funcsigs==0.4,functools32==3.2.3.post2,futures==3.0.5,futurist==0.13.0,greenlet==0.4.9,hacking==0.10.3,httplib2==0.9.2,idna==2.0,ipaddress==1.0.16,iso8601==0.1.11,Jinja2==2.8,jsonschema==2.5.1,-e git+http://git.bjv30.isscloud.com/cloudgo/keystone.git@30b5d9036f3299333a3655c00b5cd510676a48e8#egg=keystone,keystoneauth1==2.4.3,keystonemiddleware==0.0.1.dev4,kombu==3.0.34,ldappool==1.0,linecache2==1.0.0,lxml==3.5.0,MarkupSafe==0.23,mccabe==0.2.1,mock==1.3.0,monotonic==0.6,mox3==0.14.0,msgpack-python==0.4.7,netaddr==0.7.18,netifaces==0.10.4,oauthlib==1.0.3,os-client-config==1.16.0,os-testr==0.6.0,oslo.cache==1.6.0,oslo.concurrency==3.7.1,oslo.config==3.9.0,oslo.context==2.2.0,oslo.db==0.0.1.dev3,oslo.i18n==3.5.0,oslo.log==3.3.0,oslo.messaging==4.6.1,oslo.middleware==3.8.0,oslo.policy==1.6.0,oslo.serialization==2.4.0,oslo.service==1.8.0,oslo.utils==3.8.0,oslosphinx==4.3.0,oslotest==2.4.0,paramiko==1.16.0,passlib==1.6.5,Paste==2.0.2,PasteDeploy==1.5.2,pbr==1.8.1,pep257==0.7.0,pep8==1.5.7,pika==0.10.0,pika-pool==0.1.3,positional==1.0.1,pyasn1==0.1.9,pycadf==2.2.0,pycparser==2.14,pycrypto==2.6.1,pyflakes==0.8.1,Pygments==2.1.3,pyinotify==0.9.6,pymongo==3.2.1,pyOpenSSL==0.15.1,pyrsistent==0.11.12,pysaml2==4.0.2,python-dateutil==2.5.0,python-keystoneclient==0.0.1.dev40,python-ldap==2.4.25,python-memcached==1.57,python-mimeparse==1.5.1,python-subunit==1.2.0,pytz==2015.7,PyYAML==3.11,reno==2.5.0,repoze.lru==0.6,repoze.who==2.2,requests==2.9.1,requestsexceptions==1.1.3,retrying==1.3.3,Routes==2.2,six==1.10.0,Sphinx==1.2.3,SQLAlchemy==1.0.12,sqlalchemy-migrate==0.10.0,sqlparse==0.1.18,stevedore==1.12.0,tempest-lib==1.0.0,Tempita==0.5.2,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.0.0,traceback2==1.4.0,unittest2==1.1.0,waitress==0.8.10,WebOb==1.5.1,WebTest==2.0.20,wrapt==1.10.6,zope.interface==4.1.3
 py27 runtests: PYTHONHASHSEED='2134132608'
 py27 runtests: commands[0] | find keystone -type f -name *.pyc -delete
 py27 runtests: commands[1] | bash tools/pretty_tox.sh keystone.tests.unit.test_v3_assignment.AssignmentTestCase
 running testr
 running=
 ${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./keystone/tests/unit} --list

running=
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./keystone/tests/unit}  --load-list /tmp/tmpQRrBp5
running=
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./keystone/tests/unit}  --load-list /tmp/tmpkpPca8
running=
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./keystone/tests/unit}  --load-list /tmp/tmpAsqqdg
running=
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./keystone/tests/unit}  --load-list /tmp/tmpcKMYi5
{3} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_create_member_role [3.301750s] ... ok
{0} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_user_domain_role_grants_no_user [3.129583s] ... ok
{2} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_create_role [3.084101s] ... ok
{1} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_check_effective_values_for_role_assignments [3.829839s] ... ok
{3} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_create_role_bad_request [0.782877s] ... ok
{0} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_user_project_role_grants [1.012792s] ... ok
{1} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_group_domain_role_grants_no_group [0.751216s] ... ok
{2} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_group_domain_role_grants [1.185124s] ... ok
{3} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_user_domain_role_grants [1.365098s] ... ok
{2} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_group_project_role_grants_no_group [0.868791s] ... ok
{1} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_group_project_role_grants [1.128794s] ... ok
{0} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_delete_grant_from_group_and_project_invalidates_cache [1.382533s] ... ok
{3} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_user_project_role_grants_no_user [0.947176s] ... ok
{1} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_delete_grant_from_user_and_domain_invalidates_cache [0.932734s] ... ok
{2} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_delete_role [1.184809s] ... ok
{0} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_delete_user_and_check_role_assignment_fails [1.040105s] ... ok
{3} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_delete_grant_from_group_and_domain_invalidates_cache [1.143361s] ... ok
{1} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_delete_grant_from_user_and_project_invalidate_cache [1.005817s] ... ok
{0} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_get_effective_role_assignments [1.158699s] ... ok
{1} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_list_roles [0.871036s] ... ok
{2} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_filtered_role_assignments [1.769572s] ... ok
{3} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_delete_user_before_removing_role_assignment_succeeds [1.262221s] ... ok
{1} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_token_revoked_once_group_role_grant_revoked [0.880450s] ... FAILED

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "keystone/tests/unit/test_v3_assignment.py", line 336, in test_token_revoked_once_group_role_grant_revoked
        expected_status=http_client.NOT_FOUND)
      File "keystone/tests/unit/test_v3.py", line 537, in head
        expected_status=expected_status, **kwargs)
      File "keystone/tests/unit/test_v3.py", line 529, in v3_request
        return self.admin_request(path=path, token=token, **kwargs)
      File "keystone/tests/unit/rest.py", line 212, in admin_request
        return self._request(app=self.admin_app, **kwargs)
      File "keystone/tests/unit/rest.py", line 201, in _request
        response = self.restful_request(**kwargs)
      File "keystone/tests/unit/rest.py", line 186, in restful_request
        **kwargs)
      File "keystone/tests/unit/rest.py", line 90, in request
        **kwargs)
      File "/home/unittest/keystone/.tox/py27/lib/python2.7/site-packages/webtest/app.py", line 567, in request
        expect_errors=expect_errors,
      File "/home/unittest/keystone/.tox/py27/lib/python2.7/site-packages/webtest/app.py", line 632, in do_request
        self._check_status(status, res)
      File "/home/unittest/keystone/.tox/py27/lib/python2.7/site-packages/webtest/app.py", line 667, in _check_status
        "Bad response: %s (not %s)", res_status, status)
    webtest.app.AppError: Bad response: 200 OK (not 404)
    

Captured pythonlogging:
~~~~~~~~~~~~~~~~~~~~~~~
    Adding cache-proxy 'oslo_cache.testing.CacheIsolatingProxy' to backend.
    Adding cache-proxy 'oslo_cache.testing.CacheIsolatingProxy' to backend.
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4e898d0> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4b8dc90> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x46901d0> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x46901d0> acquired
    Calling creation function
    Released creation lock
    The admin_token_auth middleware presents a security risk and should be removed from the [pipeline:api_v3], [pipeline:admin_api], and [pipeline:public_api] sections of your paste ini file.
    The admin_token_auth middleware presents a security risk and should be removed from the [pipeline:api_v3], [pipeline:admin_api], and [pipeline:public_api] sections of your paste ini file.
    The admin_token_auth middleware presents a security risk and should be removed from the [pipeline:api_v3], [pipeline:admin_api], and [pipeline:public_api] sections of your paste ini file.
    The admin_token_auth middleware presents a security risk and should be removed from the [pipeline:api_v3], [pipeline:admin_api], and [pipeline:public_api] sections of your paste ini file.
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4b8fa90> acquired
    Calling creation function
    Released creation lock
    There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set.
    POST http://localhost:80/v3/auth/tokens
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x596a490> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4b8d050> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4806450> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4806450> acquired
    Calling creation function
    Released creation lock
    RBAC: Proceeding without project or domain scope
    RBAC: auth_context: {'access_token_id': None, 'user_id': u'124ec4417a804bfcaa113317e3cdf595', 'consumer_id': None, 'trustor_id': None, 'user_target': None, 'is_delegated_auth': False, 'trust_id': None, 'trustee_id': None, 'token': <KeystoneToken (audit_id=tY7TVIapRPSNoKPVgcNxfQ, audit_chain_id=tY7TVIapRPSNoKPVgcNxfQ) at 0x5e425f0>}
    POST http://localhost:80/v3/auth/tokens
    There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set.
    POST http://localhost:80/v3/auth/tokens
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4806290> acquired
    Calling creation function
    Released creation lock
    RBAC: Proceeding without project or domain scope
    RBAC: auth_context: {'access_token_id': None, 'user_id': u'124ec4417a804bfcaa113317e3cdf595', 'consumer_id': None, 'trustor_id': None, 'user_target': None, 'is_delegated_auth': False, 'trust_id': None, 'trustee_id': None, 'token': <KeystoneToken (audit_id=4jw2Y8iBSYi_34W6wseoIA, audit_chain_id=4jw2Y8iBSYi_34W6wseoIA) at 0x725c050>}
    HEAD http://localhost:80/v3/auth/tokens
    RBAC: Authorizing identity:check_token()
    RBAC: using auth context from the request environment
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x5e65810> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x5b61650> acquired
    Calling creation function
    Released creation lock
    There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set.
    POST http://localhost:80/v3/auth/tokens
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x5e83890> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x5e83f50> acquired
    Calling creation function
    Released creation lock
    RBAC: Proceeding without project or domain scope
    RBAC: auth_context: {'access_token_id': None, 'user_id': u'124ec4417a804bfcaa113317e3cdf595', 'consumer_id': None, 'trustor_id': None, 'user_target': None, 'is_delegated_auth': False, 'trust_id': None, 'trustee_id': None, 'token': <KeystoneToken (audit_id=AEqLyFtMRHG9wfKE0AVrew, audit_chain_id=AEqLyFtMRHG9wfKE0AVrew) at 0x5e1a290>}
    HEAD http://localhost:80/v3/auth/tokens
    RBAC: Authorizing identity:check_token()
    RBAC: using auth context from the request environment
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x5b61a90> acquired
    Calling creation function
    Released creation lock
    
{2} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_get_role [0.886941s] ... ok
{0} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_get_role_assignments [1.906290s] ... ok
{2} keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_update_role [0.603548s] ... ok

==============================
Failed 1 tests - output below:
==============================

keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_token_revoked_once_group_role_grant_revoked
----------------------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):
      File "keystone/tests/unit/test_v3_assignment.py", line 336, in test_token_revoked_once_group_role_grant_revoked
        expected_status=http_client.NOT_FOUND)
      File "keystone/tests/unit/test_v3.py", line 537, in head
        expected_status=expected_status, **kwargs)
      File "keystone/tests/unit/test_v3.py", line 529, in v3_request
        return self.admin_request(path=path, token=token, **kwargs)
      File "keystone/tests/unit/rest.py", line 212, in admin_request
        return self._request(app=self.admin_app, **kwargs)
      File "keystone/tests/unit/rest.py", line 201, in _request
        response = self.restful_request(**kwargs)
      File "keystone/tests/unit/rest.py", line 186, in restful_request
        **kwargs)
      File "keystone/tests/unit/rest.py", line 90, in request
        **kwargs)
      File "/home/unittest/keystone/.tox/py27/lib/python2.7/site-packages/webtest/app.py", line 567, in request
        expect_errors=expect_errors,
      File "/home/unittest/keystone/.tox/py27/lib/python2.7/site-packages/webtest/app.py", line 632, in do_request
        self._check_status(status, res)
      File "/home/unittest/keystone/.tox/py27/lib/python2.7/site-packages/webtest/app.py", line 667, in _check_status
        "Bad response: %s (not %s)", res_status, status)
    webtest.app.AppError: Bad response: 200 OK (not 404)
    

Captured pythonlogging:
~~~~~~~~~~~~~~~~~~~~~~~
    Adding cache-proxy 'oslo_cache.testing.CacheIsolatingProxy' to backend.
    Adding cache-proxy 'oslo_cache.testing.CacheIsolatingProxy' to backend.
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4e898d0> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4b8dc90> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x46901d0> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x46901d0> acquired
    Calling creation function
    Released creation lock
    The admin_token_auth middleware presents a security risk and should be removed from the [pipeline:api_v3], [pipeline:admin_api], and [pipeline:public_api] sections of your paste ini file.
    The admin_token_auth middleware presents a security risk and should be removed from the [pipeline:api_v3], [pipeline:admin_api], and [pipeline:public_api] sections of your paste ini file.
    The admin_token_auth middleware presents a security risk and should be removed from the [pipeline:api_v3], [pipeline:admin_api], and [pipeline:public_api] sections of your paste ini file.
    The admin_token_auth middleware presents a security risk and should be removed from the [pipeline:api_v3], [pipeline:admin_api], and [pipeline:public_api] sections of your paste ini file.
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4b8fa90> acquired
    Calling creation function
    Released creation lock
    There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set.
    POST http://localhost:80/v3/auth/tokens
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x596a490> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4b8d050> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4806450> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4806450> acquired
    Calling creation function
    Released creation lock
    RBAC: Proceeding without project or domain scope
    RBAC: auth_context: {'access_token_id': None, 'user_id': u'124ec4417a804bfcaa113317e3cdf595', 'consumer_id': None, 'trustor_id': None, 'user_target': None, 'is_delegated_auth': False, 'trust_id': None, 'trustee_id': None, 'token': <KeystoneToken (audit_id=tY7TVIapRPSNoKPVgcNxfQ, audit_chain_id=tY7TVIapRPSNoKPVgcNxfQ) at 0x5e425f0>}
    POST http://localhost:80/v3/auth/tokens
    There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set.
    POST http://localhost:80/v3/auth/tokens
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x4806290> acquired
    Calling creation function
    Released creation lock
    RBAC: Proceeding without project or domain scope
    RBAC: auth_context: {'access_token_id': None, 'user_id': u'124ec4417a804bfcaa113317e3cdf595', 'consumer_id': None, 'trustor_id': None, 'user_target': None, 'is_delegated_auth': False, 'trust_id': None, 'trustee_id': None, 'token': <KeystoneToken (audit_id=4jw2Y8iBSYi_34W6wseoIA, audit_chain_id=4jw2Y8iBSYi_34W6wseoIA) at 0x725c050>}
    HEAD http://localhost:80/v3/auth/tokens
    RBAC: Authorizing identity:check_token()
    RBAC: using auth context from the request environment
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x5e65810> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x5b61650> acquired
    Calling creation function
    Released creation lock
    There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set.
    POST http://localhost:80/v3/auth/tokens
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x5e83890> acquired
    Calling creation function
    Released creation lock
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x5e83f50> acquired
    Calling creation function
    Released creation lock
    RBAC: Proceeding without project or domain scope
    RBAC: auth_context: {'access_token_id': None, 'user_id': u'124ec4417a804bfcaa113317e3cdf595', 'consumer_id': None, 'trustor_id': None, 'user_target': None, 'is_delegated_auth': False, 'trust_id': None, 'trustee_id': None, 'token': <KeystoneToken (audit_id=AEqLyFtMRHG9wfKE0AVrew, audit_chain_id=AEqLyFtMRHG9wfKE0AVrew) at 0x5e1a290>}
    HEAD http://localhost:80/v3/auth/tokens
    RBAC: Authorizing identity:check_token()
    RBAC: using auth context from the request environment
    NeedRegenerationException
    no value, waiting for create lock
    value creation lock <dogpile.cache.region._LockWrapper object at 0x5b61a90> acquired
    Calling creation function
    Released creation lock
    


======
Totals
======
Ran: 26 tests in 15.0000 sec.
 - Passed: 25
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 1
Sum of execute time for each test: 37.4153 sec.

==============
Worker Balance
==============
 - Worker 0 (6 tests) => 0:00:09.655349
 - Worker 1 (7 tests) => 0:00:09.422374
 - Worker 2 (7 tests) => 0:00:09.612487
 - Worker 3 (6 tests) => 0:00:08.822696

Slowest Tests:

Test id                                                                                                               Runtime (s)
--------------------------------------------------------------------------------------------------------------------  -----------
keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_check_effective_values_for_role_assignments            3.830
keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_create_member_role                                     3.302
keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_user_domain_role_grants_no_user                   3.130
keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_create_role                                            3.084
keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_get_role_assignments                                   1.906
keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_filtered_role_assignments                              1.770
keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_delete_grant_from_group_and_project_invalidates_cache  1.383
keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_user_domain_role_grants                           1.365
keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_delete_user_before_removing_role_assignment_succeeds   1.262
keystone.tests.unit.test_v3_assignment.AssignmentTestCase.test_crud_group_domain_role_grants                          1.185
ERROR: InvocationError: '/usr/bin/bash tools/pretty_tox.sh keystone.tests.unit.test_v3_assignment.AssignmentTestCase'
___________________________________________________________ summary ___________________________________________________________
ERROR:   py27: commands failed

注意：测试的时候提示缺少.testr.conf 和.coveragerc文件 这个两个文件需要从githhub官网的openstack项目中获取，比如keystone 地址为https://github.com/openstack/keystone  切换到mitaka版本

【推广】 免费学中医，健康全家人
原文地址：https://www.cnblogs.com/web424/p/7550144.html