第九章测试与调试

第九章测试与调试 *************** 本章，我们将讨论以下话题： - Test-driven development - Dos and don'ts of writing tests - Mocking - Debugging - Logging 每个程序员都至少考虑过跳过编写测试。Django中默认的app布局拥有一个包含注释的tests.py模块。它也是测试所需的一个提示器。不过，我们经常希望跳过它。 Django中写测试和写代码很相似。实际上，它就是代码。因此，编写测试的过程看起来像是编写了两次（或者更多次）代码。有时候，我们承受了太多的压力，当我们花了很多时间尝试着去让代码正常工作时，虽然这样做看上去很荒唐。 However, eventually, it is pointless to skip tests if you ever want anyone else to use your code. Imagine that you invented an electric razor and tried to sell it to your friend saying that it worked well for you, but you haven't tested it properly. Being a good friend of yours he or she might agree, but imagine the horror if you told this to a stranger. ## 为什么要写测试软件中的测试是为了检查软件本身是否按照人们所期望的那样正常运行。假如不对软件进行测试，你或许自认为自己写的代码可以正常运行，不过你也没有办法证明软件可以正常运行。此外，重要的是要记得在Python中省略掉单元测试是比较危险的，因为Python存在自然——鸭子类型。跟Haskell这样的语言不同，类型检查在编译时并不能够严格地强制执行。单元测试在运行时得到执行（虽然是独立执行），这也是Python开发的基础。编写测试是你会体会到什么是谦逊。测试会指出你的错误，而且你也得到了一个提前修正错误到机会。实际上，是有一些人愿意在编写代码之前去写测试到。 ## 以测试驱动的开发 Test-driven development (TDD) is a for? of software develop?ent where you first write the test, run the test ?which would fail first?, and then write the ?ini?u? code needed to make the test pass. This might sound counter-intuitive. Why do we need to write tests when we know that we have not written any code and we are certain that it will fail because of that? However, look again. We do eventually write the code that ?erely satisfies these tests. This ?eans that these tests are not ordinary tests, they are ?ore like specifications. They tell you what to expect. These tests or specifications will directly co?e fro? your client's user stories. You are writing just enough code to ?ake it work. The process of test?driven develop?ent has ?any si?ilarities to the scientific ?ethod, which is the basis of ?odern science. In the scientific ?ethod, it is i?portant to fra?e the hypothesis first, gather data, and then conduct experi?ents that are repeatable and verifiable to prove or disprove your hypothesis. My recommendation would be to try TDD once you are comfortable writing tests for your projects. Beginners ?ight find it difficult to fra?e a test case that checks how the code should behave. For the same reasons, I wouldn't suggest TDD for exploratory programming. ## 一个编写测试的例子 There are different kinds of tests. However, at the minimum, a programmers need to know unit tests since they have to be able to write them. Unit testing checks the smallest testable part of an application. Integration testing checks whether these parts work well with each other. 测试存在不同的类型。不过， The word unit is the key term here. Just test one unit at a time. Let's take a look at a simple example of a test case: ```python # tests.py from django.test import TestCase from django.core.urlresolvers import resolve from .views import HomeView class HomePageOpenTestCase(TestCase): def test_home_page_resolves(self): view = resolve('/') self.assertEqual(view.func.__name__, HomeView.as_view().__name__) ``` This is a simple test that checks whether, when a user visits the root of our website's domain, they are correctly taken to the home page view. Like most good tests, it has a long and self-descriptive name. The test simply uses Django's resolve() function to match the view callable mapped to the "/" root location to the known view function by their names. It is more important to note what is not done in this test. We have not tried to retrieve the HTML contents of the page or check its status code. We have restricted ourselves to test just one unit, that is, the resolve() function, which maps the URL paths to view functions. Assuming that this test resides in, say, app1 of your project, the test can be run with the following command: ```python $ ./manage.py test app1 Creating test database for alias 'default'... . ----------------------------------------------------------------- Ran 1 test in 0.088s OK Destroying test database for alias 'default'... ``` This command runs all the tests in the app1 application or package. The default test runner will look for tests in all modules in this package matching the pattern test*.py. Django now uses the standard unittest module provided by Python rather than bundling its own. You can write a testcase class by subclassing from django.test.TestCase. This class typically has methods with the following naming convention: • test*: Any method whose name starts with test will be executed as a test method. It takes no parameters and returns no values. Tests will be run in an alphabetical order. • setUp (optional): This method will be run before each test method. It can be used to create common objects or perform other initialization tasks that bring your test case to a known state. • tearDown (optional): This method will be run after a test method, irrespective of whether the test passed or not. Clean-up tasks are usually performed here. A test case is a way to logically group test methods, all of which test a scenario. When all the test methods pass (that is, do not raise any exception), then the test case is considered passed. If any of them fail, then the test case fails. ## 断言方法 Each test method usually invokes an assert*() method to check some expected outco?e of the test. In our first exa?ple, we used assertEqual() to check whether the function name matches with the expected function. Similar to assertEqual(), the Python 3 unittest library provides more than 32 assert ?ethods. It is further extended by Django by ?ore than ?? fra?ework?specific assert ?ethods. You ?ust choose the ?ost appropriate ?ethod based on the end outcome that you are expecting so that you will get the most helpful error message. Let's see why by looking at an example testcase that has the following setUp() method: ```python def setUp(self): self.l1 = [1, 2] self.l2 = [1, 0] ``` Our test is to assert that l1 and l2 are equal (and it should fail, given their values). Let's take a look at several equivalent ways to accomplish this: 表格：略 The first state?ent uses Python's built? in assert keyword. Notice that it throws the least helpful error. You cannot infer what values or types are in the self.l1 and self.l2 variables. This is primarily the reason why we need to use the assert*() methods. Next, the exception thrown by assertEqual() very helpfully tells you that you are comparing two lists and even tells you at which position they begin to differ. This is exactly similar to the exception thrown by the more specialized assertListEqual() function. This is because, as the documentation would tell you, if assertEqual() is given two lists for comparison, then it hands it over to assertListEqual(). Despite this, as the last example proves, it is always better to use the ?ost specific assert* method for your tests. Since the second argument is not a list, the error clearly tells you that a list was expected. >?se the ?ost specific assert* method in your tests. Therefore, you need to familiarize yourself with all the assert methods, and choose the ?ost specific one to evaluate the result you expect. This also applies to when you are checking whether your application does not do things it is not supposed to do, that is, a negative test case. You can check for exceptions or warnings using assertRaises and assertWarns respectively. ## 编写更好一些的测试 We have already seen that the best test cases test a small unit of code at a time. They also need to be fast. A programmer needs to run tests at least once before every commit to the source control. Even a delay of a few seconds can tempt a programmer to skip running tests (which is not a good thing). Here are some qualities of a good test case (which is a subjective term, of course) in the form of an easy-to-remember mnemonic "F.I.R.S.T. class test case": 1. Fast: the faster the tests, the more often they are run. Ideally, your tests should complete in a few seconds. 2. Independent: Each test case must be independent of others and can be run in any order. 3. Repeatable: The results must be the same every time a test is run. Ideally, all random and varying factors must be controlled or set to known values before a test is run. 4. Small: Test cases must be as short as possible for speed and ease of understanding. 5. Transparent: Avoid tricky implementations or ambiguous test cases. Additionally, make sure that your tests are automatic. Eliminate any manual steps, no matter how small. Automated tests are more likely to be a part of your team's work?ow and easier to use for tooling purposes. Perhaps, even more important are the don'ts to remember while writing test cases: • Do not (re)test the framework: Django is well tested. Don't check for URL lookup, template rendering, and other framework-related functionality. • Do not test implementation details: Test the interface and leave the minor implementation details. It makes it easier to refactor this later without breaking the tests. • Test models most, templates least: Templates should have the least business logic, and they change more often. • Avoid HTML output validation: Test views use their context variable's output rather than its HTML-rendered output. • Avoid using the web test client in unit tests: Web test clients invoke several components and are therefore, better suited for integration tests. • Avoid interacting with external systems: Mock them if possible. Database is an exception since test database is in-memory and quite fast. Of course, you can (and should) break the rules where you have a good reason to ?just like I did in ?y first exa?ple?. ?lti?ately, the ?ore creative you are at writing tests, the earlier you can catch bugs, and the better your application will be. ## Mocking Most real-life projects have various interdependencies between components. While testing one component, the result must not be affected by the behavior of other components. For example, your application might call an external web service that might be unreliable in terms of network connection or slow to respond. Mock objects imitate such dependencies by having the same interface, but they respond to method calls with canned responses. After using a mock object in a test, you can assert whether a certain method was called and verify that the expected interaction took place. Take the exa?ple of the ?uperHero profile eligibility test ?entioned in Pattern: Service objects (see Chapter 3, Models). We are going to mock the call to the service object method in a test using the Python 3 unittest.mock library: ```python # profiles/tests.py from django.test import TestCase from unittest.mock import patch from django.contrib.auth.models import User class TestSuperHeroCheck(TestCase): def test_checks_superhero_service_obj(self): with patch("profiles.models.SuperHeroWebAPI") as ws: ws.is_hero.return_value = True u = User.objects.create_user(username="t") r = u.profile.is_superhero() ws.is_hero.assert_called_with('t') self.assertTrue(r) ``` Here, we are using patch() as a context manager in a with statement. Since the profile ?odel's is_superhero() method will call the SuperHeroWebAPI.is_hero() class method, we need to mock it inside the models module. We are also hard-coding the return value of this method to be True. The last two assertions check whether the method was called with the correct arguments and if is_hero() returned True, respectively. Since all methods of SuperHeroWebAPI class have been mocked, both the assertions will pass. Mock objects come from a family called Test Doubles, which includes stubs, fakes, and so on. Like movie doubles who stand in for real actors, these test doubles are used in place of real objects while testing. While there are no clear lines drawn between them, Mock objects are objects that can test the behavior, and stubs are simply placeholder implementations. ##Pattern ? test fi?tures ?and f?actories Problem: Testing a component requires the creation of various prerequisite objects before the test. Creating them explicitly in each test method gets repetitive. Solution: Utilize factories or fixtures to create the test data objects. ## 问题细节 Before running each test, Django resets the database to its initial state, as it would be after running migrations. Most tests will need the creation of some initial objects to set the state. Rather than creating different initial objects for different scenarios, a common set of initial objects are usually created. This can quickly get unmanageable in a large test suite. The sheer variety of such initial objects can be hard to read and later understand. This leads to hard?to?find bugs in the test data itself! Being such a common problem, there are several means to reduce the clutter and write clearer test cases. ## 解决方法细节 The first solution we will take a look at is what is given in the Django docu?entation itself?test fixtures. Here, a test fixture is a file that contains a set of data that can be i?ported into your database to bring it to a known state. Typically, they are Y?ML or ???? files previously exported from the same database when it had some data. For exa?ple, consider the following test case, which uses a test fixture?: ```python from django.test import TestCase class PostTestCase(TestCase): fixtures = ['posts'] def setUp(self): # Create additional common objects pass def test_some_post_functionality(self): # By now fixtures and setUp() objects are loaded pass ``` Before setUp() gets called in each test case, the specified fixture, posts gets loaded. Roughly speaking, the fixture would be searched for in the fixtures directory with certain known extensions, for example, app/fixtures/posts.json. However, there are a nu?ber of proble?s with fixtures. Fixtures are static snapshots of the database. They are schema-dependent and have to be changed each time your models change. They also might need to be updated when your test-case assertions change. ?pdating a large fixture file ?anually, with ?ultiple related objects, is no joke. For all these reasons, ?any consider using fixtures as an anti?pattern. It is recommended that you use factories instead. A factory class creates objects of a particular class that can be used in tests. It is a DRY way of creating initial test objects. Let's use a model's objects.create method to create a simple factory: ```python from django.test import TestCase from .models import Post class PostFactory: def make_post(self): return Post.objects.create(message="") class PostTestCase(TestCase): def setUp(self): self.blank_message = PostFactory().makePost() def test_some_post_functionality(self): pass ``` ?o?pared to using fixtures, the initial object creation and the test cases are all in one place. Fixtures load static data as is into the database without calling ?odel?defined save() methods. Since factory objects are dynamically generated, they are more likely to run through your application's custom validations. However, there is a lot of boilerplate in writing such factory classes yourself. The factory_boy package, based on thoughtbot's factory_girl, provides a declarative syntax for creating object factories. Rewriting the previous code to use factory_boy, we get the following result: ```python import factory from django.test import TestCase from .models import Post class PostFactory(factory.Factory): class Meta: model = Post message = "" class PostTestCase(TestCase): def setUp(self): self.blank_message = PostFactory.create() self.silly_message = PostFactory.create(message="silly") def test_post_title_was_set(self): self.assertEqual(self.blank_message.message, "") self.assertEqual(self.silly_message.message, "silly") ``` Notice how clear the `factory` class becomes when written in a declarative fashion. The attribute's values do not have to be static. You can have sequential, rando?, or computed attribute values. If you prefer to have more realistic placeholder data such as US addresses, then use the `django-faker` package. In conclusion, I would recommend factories, especially factory_boy, for most projects that need initial test objects. One ?ight still want to use fixtures for static data, such as lists of countries or t-shirt sizes, since they would rarely change. >#####Dire Predictions After the announcement of the impossible deadline, the entire team seemed to be suddenly out of time. They went from 4-week scrum sprints to 1-week sprints. Steve wiped every meeting off their calendars except "today's 30-minute catch-up with Steve." He preferred to have a one-on-one discussion if he needed to talk to someone at their desk. >At Madam O's insistence, the 30-minute meetings were held at a sound proof hall 20 levels below the S.H.I.M. headquarters. On Monday, the team stood around a large circular table with a gray metallic surface like the rest of the room. Steve stood awkwardly in front of it and made a stiff waving gesture with an open palm. >Even though everyone had seen the holographs come alive before, it never failed to amaze them each time. The disc almost segmented itself into hundreds of metallic squares and rose like miniature skyscrapers in a futuristic model city. It took them a second to realize that they were looking at a 3D bar chart. >"Our burn-down chart seems to be showing signs of slowing down. I am guessing it is the outcome of our recent user tests, which is a good thing. But..." Steve's face seemed to show the strain of trying to sti?e a snee?e. He gingerly ?icked his forefinger upwards in the air and the chart smoothly extended to the right. >"At this rate, projections indicate that we will miss the go-live by several days, at best. I did a bit of analysis and found several critical bugs late in our development. We can save a lot of time and effort if we can catch them early. I want to put your heads together and come up with some i..." >Steve clasped his mouth and let out a loud sneeze. The holograph interpreted this as a sign to zoom into a particularly uninteresting part of the graph. Steve cursed under his breath and turned it off. He borrowed a napkin and started noting down everyone's suggestions with an ordinary pen. >One of the suggestions that Steve liked most was a coding checklist listing the most common bugs, such as forgetting to apply migrations. He also liked the idea of involving users earlier in the development process for feedback. He also noted down some unusual ideas, such as a Twitter handle for tweeting the status of the continuous integration server. >At the close of the meeting, Steve noticed that Evan was missing. ?Where is ?van?? he asked. ??o idea,? said Brad looking confused, "he was here a minute ago." ##Learning more about testing Django's default test runner has improved a lot over the years. However, test runners such as py.test and nose are still superior in terms of functionality. They make your tests easier to write and run. Even better, they are compatible with your existing test cases. You ?ight also be interested in knowing what percentage of your code is covered by tests. This is called Code coverage and coverage.py is a very popular tool for finding this out. Most projects today tend to use a lot of JavaScript functionality. Writing tests for them usually require a browser-like environment for execution. Selenium is a great browser automation tool for executing such tests. While a detailed treatment of testing in Django is outside the scope of this book, I would strongly recommend that you learn more about it. If nothing else, the two main takeaways I wanted to convey through this section are first, write tests, and second, once you are confident at writing the?, practice TDD. ##Debugging Despite the most rigorous testing, the sad reality is, we still have to deal with bugs. Django tries its best to be as helpful as possible while reporting an error to help you in debugging. However, it takes a lot of skill to identify the root cause of the problem. Thankfully, with the right set of tools and techniques, we can not only identify the bugs but also gain great insight into the runtime behavior of your code. Let's take a look at some of these tools. ## Django的调试页面 If you have encountered any exception in development, that is, when DEBUG=True, then you would have already seen an error page similar to the following screenshot: 图片：略 Since it comes up so frequently, most developers tend to miss the wealth of information in this page. Here are some places to take a look at: ``` • Exception details: Obviously, you need to read what the exception tells you very carefully. • Exception location: This is where Python thinks where the error has occurred. In Django, this may or may not be where the root cause of the bug is. • Traceback: This was the call stack when the error occurred. The line that caused the error will be at the end. The nested calls that led to it will be above it. Don't forget to click on the 'Local vars' arrow to inspect the values of the variables at the time of the exception. • Request information: This is a table (not shown in the screenshot) that shows context variables, meta information, and project settings. Check for malformed input in the requests here. ``` ## 更友好的测试页面 Often, you may wish for more interactivity in the default Django error page. The django-extensions package ships with the fantastic Werkzeug debugger that provides exactly this feature. In the following screenshot of the same exception, notice a fully interactive Python interpreter available at each level of the call stack: 图片：略 To enable this, in addition to adding django_extensions to your INSTALLED_APPS, you will need to run your test server as follows: ```python $ python manage.py runserver_plus ``` Despite the reduced debugging infor?ation, I find the Werk?eug debugger to be more useful than the default error page. ## print函数 Sprinkling print() functions all over the code for debugging might sound primitive, but it has been the preferred technique for many programmers. Typically, the print() functions are added before the line where the exception has occurred. It can be used to print the state of variables in various lines leading to the exception. You can trace the execution path by printing so?ething when a certain line is reached. In development, the print output usually appears in the console window where the test server is running. Whereas in production, these print outputs might end up in your server log file where they would add a runti?e overhead. In any case, it is not a good debugging technique to use in production. Even if you do, the print functions that are added for debugging should be removed from being committed to your source control. ## 日志 The ?ain reason for including the previous section was to say?You should replace the print() functions with calls to logging functions in Python's logging module. Logging has several advantages over printing: it has a timestamp, a clearly marked level of urgency (for example, INFO, DEBUG), and you don't have to remove them from your code later. Logging is fundamental to professional web development. Several applications in your production stack, like web servers and databases, already use logs. Debugging might take you to all these logs to retrace the events that lead to a bug. It is only appropriate that your application follows the same best practice and adopts logging for errors, warnings, and informational messages. Unlike the common perception, using a logger does not involve too much work. Sure, the setup is slightly involved but it is merely a one-time effort for your entire project. Even more, most project templates (for example, the edge template) already do this for you. ?nce you have configured the LOGGING variable in settings.py, adding a logger to your existing code is quite easy, as shown here: ```python # views.py import logging logger = logging.getLogger(__name__) def complicated_view(): logger.debug("Entered the complicated_view()!") ``` The logging module provides various levels of logged messages so that you can easily filter out less urgent ?essages. The log output can be also formatted in various ways and routed to ?any places, such as standard output or log files. Read the documentation of Python's logging module to learn more. ## Django调试工具 The Django Debug Toolbar is an indispensable tool not just for debugging but also for tracking detailed information about each request and response. Rather than appearing only during exceptions, the toolbar is always present in your rendered page. Initially, it appears as a clickable graphic on the right-hand side of your browser window. On clicking, a toolbar appears as a dark semi-transparent sidebar with several headers: 图片：略 ?ach header is filled with detailed infor?ation about the page fro? the nu?ber of SQL queries executed to the templates that we use to render the page. Since the toolbar disappears when DEBUG is set to False, it is pretty much restricted to being a development tool. The Python debugger pdb While debugging, you might need to stop a Django application in the middle of execution to examine its state. A simple way to achieve this is to raise an exception with a simple assert False line in the required place. What if you wanted to continue the execution step by step fro? that line? This is possible with the use of an interactive debugger such as Python's pdb. Simply insert the following line wherever you want the execution to stop and switch to pdb: ```python import pdb; pdb.set_trace() ``` Once you enter pdb, you will see a command-line interface in your console window with a (Pdb) prompt. At the same time, your browser window will not display anything as the request has not finished processing. The pdb command-line interface is extremely powerful. It allows you to go through the code line by line, examine the variables by printing them, or execute arbitrary code that can even change the running state. The interface is quite similar to GDB, the GNU debugger. ## 其他的调试器 There are several drop-in replacements for pdb. They usually have a better interface. Some of the console-based debuggers are as follows: • ipdb: Like IPython, this has autocomplete, syntax-colored code, and so on. • pudb: Like old Turbo C IDEs, this shows the code and variables side by side. • IPython: This is not a debugger. You can get a full IPython shell anywhere in your code by adding the from IPython import embed; embed()line. PuDB is my preferred replacement for pdb. It is so intuitive that even beginners can easily use this interface. Like pdb, just insert the following code to break the execution of the program: ```python import pudb; pudb.set_trace() ``` When this line is executed, a full-screen debugger is launched, as shown here: 图片：略 Press the ? key to get help on the complete list of keys that you can use. 此外，还存在多种图形化的调试器，有些是独立使用的，比如winpdb和其他的集成到IDE中的调试器，比如PyCharm,PyDev,和Komodo。我们会推荐你多去尝试几个调试器，直道你发现那个适合你的工具。 ## 调试Django模板在项目中模板会存在非常复杂的逻辑。在创建模板时细微的bug会引发难以发现的不过。我们需要在`settings.py` 中设置TEMPLATE_DEBUG为True，这样Django就可以更好在模板中出现错误时显示错误页面。有多种简陋的方法来调试模板，比如插入想要的变量，{{ variable }}，或者是你想要像这样使用内建的调试标签，拉取所有的变量（在一个很方便的，可点击的文本区域内部）： ```python <textarea onclick="this.focus();this.select()" style="width: 100%;"> {% filter force_escape %} {% debug %} {% endfilter %} </textarea> ``` 更好的选择是使用之前提到的Django调试工具。它不仅能够告诉你上下文变量中的值，而且还显示了模板中的继承树。不过，你要是想要在模板中部暂停以检查状态（即，在循环内部），调试将会是此类情况的最佳选择。实际上，你可以使用之前提到Python解释器中的任何一个来为你的模板使用自定义模板标签。下面是一个此类模板标签的简单实现。在模板标签的包目录内创建以下文件： ```python # templatetags/debug.py import pudb as dbg # Change to any *db from django.template import Library, Node register = Library() class PdbNode(Node): def render(self, context): dbg.set_trace() return '' @register.tag def pdb(parser, token): # Debugger will stop here return PdbNode() ``` 在模板中，载入模板标签库，在需要执行暂停的地方插入pdg标签，以及输入调试器： ```python {% load debug %} {% for item in items %} {# Some place you want to break #} {% pdb %} {% endfor %} ``` 在调试器内部，你可以验证任何事情，使用上下文字典引入上下文变量： ```python >>> print(context["item"]) Item0 ``` 如果你需要在调试和内省上使用更多的模板标签，那么我会推荐你试验下`django-template-debug`包。 ## 总结这一章，我们浏览了Django中执行测试的背后动机和概念。我们也发现了在编写测试时所遵循的多种最佳实践。在调试的小节，我们熟悉了使用多种调试工具和技术发现在Django代码和模板的问题。在下一章，我们会通过理解多种安全问题，以及如何减少来自各种恶意攻击的威胁，来更进一步靠近生产环境中的代码。

第九章 测试与调试