Python Unit Testing

开始看Openstack中的单元测试，其中Mock用的比较多。记得在Openstack的某个wiki上看到建议用Mock，少用Mox。现在社区有patch逐步将mox替换为mock。

Openstack单元测试中主要由两种形式(不准确哦)：

1. @mock.patch(….)
2. @mock.patch.object(…)

这篇文章对两者有说明。回头看看。

Mock已经成为了Python 3.3标准库的一部分，文档在这里。Python 2.x中需要import mock。

开始

先来看Mock官方文档：
http://www.voidspace.org.uk/python/mock/index.html

mock provides a core Mock class removing the need to create a host of stubs throughout your test suite. After performing an action, you can make assertions about which methods / attributes were used and arguments they were called with. You can also specify return values and set needed attributes in the normal way.

Additionally, mock provides a patch() decorator that handles patching module and class level attributes within the scope of a test, along with sentinel for creating unique objects. See the quick guide for some examples of how to use Mock, MagicMock and patch().

Mock is very easy to use and is designed for use with unittest. Mock is based on the ‘action -> assertion’ pattern instead of ‘record -> replay’ used by many mocking frameworks.

Quick start guide

这里的patch(),patch.object()用法：

patch()装饰器可以很容易的模拟被测模块中的class/object：

>>> from mock import patch>>> @patch('module.ClassName2')... @patch('module.ClassName1')... def test(MockClass1, MockClass2):...     module.ClassName1()...     module.ClassName2()...     assert MockClass1 is module.ClassName1...     assert MockClass2 is module.ClassName2...     assert MockClass1.called...     assert MockClass2.called...>>> test()

这个例子用mock模拟了module中的两个类ClassName1、ClassName2。
细节不明白：
test中实例化了两个对象，为什么MockClass1==module.ClassName1，并且MockClass.called=True呢？

【看完下面的例一就明白了。这里修饰了两次，所以有test方法在定义时多了两个参数，分别表示ClassName1, ClassName2】

patch.object()在with中的用法：

>>> with patch.object(ProductionClass, 'method', return_value=None) as mock_method:...     thing = ProductionClass()...     thing.method(1, 2, 3)...>>> mock_method.assert_called_once_with(1, 2, 3)

这个例子模拟了ProductionClass中的method方法，让这个方法的调用参数为(1, 2, 3)，最后用assert_called_once_with判断这个method被调用了一次且参数为(1, 2, 3)。

单看上面的例子不是很明白，下面看看patch的说明

Patch()

总的来说，patch()用一个mock.MagicMoc对象模拟了某个class，可以用mock.MagicMoc对象的方法控制class实例化后的行为，比如可以控制class中某个方法的返回值（例二）。

文档在这里

def patch(        target, new=DEFAULT, spec=None, create=False,        spec_set=None, autospec=None, new_callable=None, **kwargs    ):    """    `patch` acts as a function decorator, class decorator or a context    manager. Inside the body of the function or with statement, the `target`    is patched with a `new` object. When the function/with statement exits    the patch is undone.

摘几个说明：

target should be a string in the form ‘package.module.ClassName’. The target is imported and the specified object replaced with the new object, so the target must be importable from the environment you are calling patch from. The target is imported when the decorated function is executed, not at decoration time.

• 这个类package.module.ClassName 在被import时会被Mock对象模拟
• 在当前环境下这个类一定要可以被import

Patch can be used as a context manager, with the with statement. Here the patching applies to the indented block after the with statement. If you use as then the patched object will be bound to the name after the as; very useful if patch is creating a mock object for you.

从例子中理解:

例一

patch修饰方法：

# test1.pyfrom mock import patch@patch('__main__.SomeClass')def function(normal_arg, mock_class_123):    a = SomeClass('name')    print "mock_class_123 is SomeClass: %r\n" % (mock_class_123 is SomeClass)    print "dir(mock_class_123): \n%r\n" % dir(mock_class_123)    print "type(a): %r\n" % type(a)    print "a.arg: %r\n" % a.arg    print "dir(a): \n%r" % dir(a)class SomeClass(object):    def __init__(self, arg):        super(SomeClass, self).__init__()        self.arg = argfunction(123)

felix@ubuntu14-home:~/work/practise/mock/test-patch-function|⇒  python test1.pymock_class_123 is SomeClass: Truedir(mock_class_123):['assert_any_call', 'assert_called_once_with', 'assert_called_with', 'assert_has_calls', 'attach_mock', 'call_args', 'call_args_list', 'call_count', 'called', 'configure_mock', 'method_calls', 'mock_add_spec', 'mock_calls', 'reset_mock', 'return_value', 'side_effect']type(a): <class 'mock.MagicMock'>a.arg: <MagicMock name='SomeClass().arg' id='139839094195280'>dir(a):['arg', 'assert_any_call', 'assert_called_once_with', 'assert_called_with', 'assert_has_calls', 'attach_mock', 'call_args', 'call_args_list', 'call_count', 'called', 'configure_mock', 'method_calls', 'mock_add_spec', 'mock_calls', 'reset_mock', 'return_value', 'side_effect']

• function中用到了SomeClass，用patch修饰function可以伪造SomeClass
• 这种情况下，定义function时，需要在最后多一个参数，该参数为mock.MagicMock对象（参数名任意）。如果function定义改为def function(normal_arg)，执行function(123)报错TypeError: function() takes exactly 1 argument (2 given)
• 被模拟的class是一个mock.MagicMock对象

例二

patch放在with...as...中
（实际上就是把例一中function中最后的参数mock_class_123放在了as中）

  1 # test2.py  2 from mock import patch  3  4 class SomeClass(object):  5     def __init__(self, arg):  6         super(SomeClass, self).__init__()  7         self.arg = arg  8  9 with patch('__main__.SomeClass') as MockSomeClass: 10     import ipdb;ipdb.set_trace() 11     instance = MockSomeClass.return_value 12     instance.method.return_value = 'foo' 13     assert SomeClass() is instance 14     assert SomeClass().method() == 'foo'

• 第9行把SomeClass模拟了，构造出的MagicMock对象赋值给MockSomeClass

ipdb> MockSomeClass<MagicMock name='SomeClass' id='140198820489936'>ipdb> MockSomeClass.MockSomeClass.assert_any_call          MockSomeClass.call_args_list           MockSomeClass.mock_callsMockSomeClass.assert_called_once_with  MockSomeClass.call_count               MockSomeClass.reset_mockMockSomeClass.assert_called_with       MockSomeClass.called                   MockSomeClass.return_valueMockSomeClass.assert_has_calls         MockSomeClass.configure_mock           MockSomeClass.side_effectMockSomeClass.attach_mock              MockSomeClass.method_callsMockSomeClass.call_args                MockSomeClass.mock_add_spec

• 第11行把MockSomeClass.return_value赋值给instance。MockSomeClass.return_value实际上也是MagicMock对象，但是比MockSomeClass具有更多属性，因为它是”return_value”

ipdb> type(MockSomeClass.return_value)<class 'mock.MagicMock'>ipdb> MockSomeClass.return_value.MockSomeClass.return_value.assert_any_call          MockSomeClass.return_value.calledMockSomeClass.return_value.assert_called_once_with  MockSomeClass.return_value.configure_mockMockSomeClass.return_value.assert_called_with       MockSomeClass.return_value.method_callsMockSomeClass.return_value.assert_has_calls         MockSomeClass.return_value.mock_add_specMockSomeClass.return_value.attach_mock              MockSomeClass.return_value.mock_callsMockSomeClass.return_value.call_args                MockSomeClass.return_value.reset_mockMockSomeClass.return_value.call_args_list           MockSomeClass.return_value.return_valueMockSomeClass.return_value.call_count               MockSomeClass.return_value.side_effect

如果在第11行之后，加一行inst = MockSomeClass.return_value，那么inst和instance完全一样：

ipdb> inst = MockSomeClass.return_valueipdb> inst<MagicMock name='SomeClass()' id='140198743303056'>ipdb> instance<MagicMock name='SomeClass()' id='140198743303056'>ipdb> inst == instanceTrueipdb> id(inst)140198743303056ipdb> id(instance)140198743303056

• 注意，第12行以前，instance没有method属性
• 第12行很有意思，instance.method.return_value = 'foo'，执行完以后，instance多了一个method属性，并且instance.method也是一个MagicMock对象：

ipdb> type(instance.method)<class 'mock.MagicMock'>ipdb> instance.meinstance.method        instance.method_calls       # 多了method属性ipdb> instance.method.instance.method.assert_any_call          instance.method.call_args_list           instance.method.mock_callsinstance.method.assert_called_once_with  instance.method.call_count               instance.method.reset_mockinstance.method.assert_called_with       instance.method.called                   instance.method.return_valueinstance.method.assert_has_calls         instance.method.configure_mock           instance.method.side_effectinstance.method.attach_mock              instance.method.method_calls             instance.method.trait_namesinstance.method.call_args                instance.method.mock_add_spec

其实这个method就是SomeClass的一个方法，第12行实际上模拟了SomeClass中名字为method的方法，把它的返回值都设置成了foo
- 第13行，SomeClass()实例化了SomeClass，而第11行把SomeClass的return_value赋值给了instance，所以assert为True。对inst同样有效：

ipdb> SomeClass() is instTrue

• 第14行，assert SomeClass().method() == 'foo'，因为第12行把SomeClass()的method()的返回值设置成了foo，所以assert成功。需要注意，这个例子中的method是不带参数的哦。【带参数怎么办？】

patch.object()

它是模拟某个class的某个method

patch.object(target, attribute, new=DEFAULT, spec=None, create=False,               spec_set=None, autospec=None, new_callable=None, **kwargs)patch the named member (`attribute`) on an object (`target`) with a mock  object.`patch.object` can be used as a decorator, class decorator or a contextmanager. Arguments `new`, `spec`, `create`, `spec_set`,`autospec` and `new_callable` have the same meaning as for `patch`. Like`patch`, `patch.object` takes arbitrary keyword arguments for configuringthe mock object it creates.

You can either call patch.object with three arguments or two arguments. The three argument form takes the object to be patched, the attribute name and the object to replace the attribute with.

When calling with the two argument form you omit the replacement object, and a mock is created for you and passed in as an extra argument to the decorated function

例三

# test-3.pyfrom mock import patchclass SomeClass(object):    def __init__(self, arg):        super(SomeClass, self).__init__()        self.arg = arg    def class_method():      [e-mail](SomeClass, 'class_method')def test(mock_method):    import ipdb;ipdb.set_trace()    SomeClass.class_method(3)    mock_method.assert_called_with(3)test()

SomeClass有一个方法class_method，test方法会调用这个class_method。
用@patch.object(SomeClass, 'class_method')修饰test方法之后：

• 查看SomeClass和SomeClass.class_method

ipdb> SomeClass.SomeClass.class_method  SomeClass.mroipdb> type(SomeClass)<type 'type'>ipdb> dir(SomeClass)['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'class_method']ipdb> type(SomeClass.class_method)<class 'mock.MagicMock'>ipdb> type(SomeClass.mro)<type 'builtin_function_or_method'>ipdb> SomeClass.class_method.SomeClass.class_method.assert_any_call          SomeClass.class_method.calledSomeClass.class_method.assert_called_once_with  SomeClass.class_method.configure_mockSomeClass.class_method.assert_called_with       SomeClass.class_method.method_callsSomeClass.class_method.assert_has_calls         SomeClass.class_method.mock_add_specSomeClass.class_method.attach_mock              SomeClass.class_method.mock_callsSomeClass.class_method.call_args                SomeClass.class_method.reset_mockSomeClass.class_method.call_args_list           SomeClass.class_method.return_valueSomeClass.class_method.call_count               SomeClass.class_method.side_effect

1）SomeClass的属性没变
2）SomeClass.class_method是一个mock.MagicMock对象

• 查看mock_method

ipdb> mock_method<MagicMock name='class_method' id='139747337265424'>ipdb> SomeClass<class '__main__.SomeClass'>ipdb> SomeClass.class_method<MagicMock name='class_method' id='139747337265424'>ipdb> SomeClass.class_method is mock_methodTrue

mock_method和SomeClass.class_method完全一样

• 注意，和patch()修饰function一样，这里的test方法在调用时没有传入参数，而在定义时有一个参数mock_method。即：修饰一次多一个参数（也就是mock.MagicMock对象）。

nested patch

例四

Like all context-managers patches can be nested using contextlib’s nested function; every patching will appear in the tuple after “as”:

一一对应：

>>> from contextlib import nested>>> with nested(...         patch('package.module.ClassName1'),...         patch('package.module.ClassName2')...     ) as (MockClass1, MockClass2):...     assert package.module.ClassName1 is MockClass1...     assert package.module.ClassName2 is MockClass2...

mock.MagicMock()

Openstack单元测试中用的也很多。 — TBD

除了mock官方文档，python 3.3 unittest.mock的文档中的例子也很好。稍后看看。

使用误区

读完上面这些例子后，估计看代码问题不大，但使用时不一定很顺。mock文档中提到了where-to-patch 还不是很明白。这个blog有几个比较好的例子。
这里先把where-to-patch 列出来：

The basic principle is that you patch where an object is looked up, which is not necessarily the same place as where it is defined. A couple of examples will help to clarify this.

Imagine we have a project that we want to test with the following structure:

a.py
-> Defines SomeClass

b.py
-> from a import SomeClass
-> some_function instantiates SomeClass

Now we want to test some_function but we want to mock out SomeClass using patch. The problem is that when we import module b, which we will have to do then it imports SomeClass from module a. If we use patch to mock out a.SomeClass then it will have no effect on our test; module b already has a reference to the real SomeClass and it looks like our patching had no effect.

The key is to patch out SomeClass where it is used (or where it is looked up ). In this case some_function will actually look up SomeClass in module b, where we have imported it. The patching should look like:

@patch('b.SomeClass')

However, consider the alternative scenario where instead of from a import SomeClass module b does import a and some_function uses a.SomeClass. Both of these import forms are common. In this case the class we want to patch is being looked up on the a module and so we have to patch a.SomeClass instead:

@patch('a.SomeClass')

我的理解：

1. 第一种方式，b.py以from a import SomeClass的方式导入SomeClass，此时在b.py的命名空间内没有a，只有SomeClass，所以patch的时候用@patch('b.SomeClass')
2. 第二种方式，b.py以import a的方式导入，此时b.py的命名空间内只有a，没有SomeClass，但是可以用a.SomeClass访问SomeClass，所以patch的时候用@patch('a.SomeClass')

其他

1. side_effect

>>> import mock>>> m = mock.Mock()>>> m.some_method.return_value = 42 # some_method并没有定义# return_value是Mock对象的一个attribute，它指定了some_method的expected返回值是42>>> dir(m.some_method)['assert_any_call', 'assert_called_once_with', 'assert_called_with', 'assert_has_calls', 'attach_mock', 'call_args', 'call_args_list', 'call_count', 'called', 'configure_mock', 'method_calls', 'mock_add_spec', 'mock_calls', 'reset_mock', 'return_value', 'side_effect']>>> m.some_method<Mock name='mock.some_method' id='25408848'>>>> m.some_method()42>>> m.some_method.return_value42>>> def print_hello():... print "hello!"...>>> m.some_method.side_effect = print_hello# side_effect指定了运行m.some_method()实际要运行的code>>> m.some_method()hello!>>> m.some_method.side_effect<function print_hello at 0x18c4050>>>> m.some_method.side_effect()hello!>>> def print_hello():... print "hello!"... return 43...>>> m.some_method()hello!>>> m.some_method.side_effect = print_hello>>> m.some_method()hello!43>>> m.some_method.call_count  # call_count记录了m.some_method被调用的次数4>>>

参考

1. http://www.voidspace.org.uk/python/mock/index.html
2. https://docs.python.org/3/library/unittest.mock.html
3. http://alexmarandon.com/articles/python_mock_gotchas/
4. http://blog.csdn.net/juvxiao/article/details/21562325