@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.)
hq.recaptime.dev/wiki/Phorge
phorge
phabricator
1@title Differential User Guide: Test Plans
2@group userguide
3
4This document describes things you should think about when developing a test
5plan.
6
7= Overview =
8
9When you send a revision for review in Differential you must include a test plan
10(this can be disabled or made optional in the config). A test plan is a
11repeatable list of steps which document what you have done to verify the
12behavior of a change. A good test plan convinces a reviewer that you have been
13thorough in making sure your change works as intended and has enough detail to
14allow someone unfamiliar with your change to verify its behavior.
15
16This document has some common things to think about when developing or reviewing
17a test plan. Some of these suggestions might not be applicable to the software
18you are writing; they are adapted from Facebook's internal documentation.
19
20= All Changes =
21
22 - **Error Handling:** Are errors detected and handled properly? How does your
23 change deal with error cases? Did you test them and make sure you got the
24 right error messages and the right behavior? It's important that you test
25 what happens when things go wrong, not just that the change works if
26 everything else also works.
27 - **Service Impact:** How does your change affect services like memcache,
28 thrift, or databases? Are you adding new cachekeys or queries? Will
29 this change add a lot of load to services?
30 - **Performance:** How does your change affect performance? **NOTE**: If
31 your change is a performance-motivated change, you should include
32 measurements, profiles or other data in your test plan proving that you have
33 improved performance.
34 - **Unit Tests:** Is your change adequately covered by unit tests? Could you
35 improve test coverage? If you're fixing a bug, did you add a test to prevent
36 it from happening again? Are the unit tests testing just the code in
37 question, or would a failure of a database or network service cause your
38 test to fail?
39 - **Concurrent Change Robustness:** If you're making a refactoring change, is
40 it robust against people introducing new calls between the time you started
41 the change and when you commit it? For example, if you change the parameter
42 order of some function from `f(a, b)` to `f(b, a)` and a new callsite is
43 introduced in the meantime, could it go unnoticed? How bad would that be?
44 (Because of this risk, you should almost never make parameter order
45 changes in weakly typed languages like PHP and Javascript.)
46 - **Revert Plan:** If your change needs to be reverted and you aren't around,
47 are any special steps or considerations that the reverter needs to know
48 about? If there are, make sure they're adequately described in the "Revert
49 Plan" field so someone without any knowledge of your patch (but with a
50 general knowledge of the system) can successfully revert your change.
51 - **Security:** Is your change robust against XSS, CSRF, and injection
52 attacks? Are you verifying the user has the right capabilities or
53 permissions? Are you consistently treating user data as untrustworthy? Are
54 you escaping data properly, and using dangerous functions only
55 when they are strictly necessary?
56 - **Architecture:** Is this the right change? Could there be a better way to
57 solve the problem? Have you talked to (or added as reviewers) domain experts
58 if you aren't one yourself? What are the limitations of this solution? What
59 tradeoffs did you make, and why?
60
61= Frontend / User-Facing Changes =
62
63 - **Static Resources:** Will your change cause the application to serve more
64 JS or CSS? Can you use less JS/CSS, or reuse more?
65 - **Browsers:** Have you tested your change in multiple browsers?