@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.)
hq.recaptime.dev/wiki/Phorge
phorge
phabricator
1@title Providing Reproduction Steps
2@group detail
3
4Describes how to provide reproduction steps.
5
6Overview
7========
8
9When you submit a bug report about Phorge, you **MUST** include
10reproduction steps. We can not help you with bugs we can not reproduce, and
11will not accept reports which omit reproduction steps or have incomplete or
12insufficient instructions.
13
14This document explains what we're looking for in good reproduction steps.
15Briefly:
16
17 - Reproduction steps must allow us to reproduce the problem locally on a
18 clean, up-to-date install of Phorge.
19 - Reproduction should be as simple as possible.
20
21Good reproduction steps can take time to write out clearly, simplify, and
22verify. As a reporter, we expect you to shoulder as much of this burden as you
23can, to make make it easy for us to reproduce and resolve bugs.
24
25We do not have the resources to pursue reports with limited, inaccurate, or
26incomplete reproduction steps, and will not accept them. These reports require
27large amounts of our time and are often fruitless.
28
29
30Example Reproduction Steps
31==========================
32
33Here's an example of what good reproduction steps might look like:
34
35---
36
37Reproduction steps:
38
39 - Click "Create Event" in Calendar.
40 - Fill in the required fields with any text (name, description, etc).
41 - Set an invalid time for one of the dates, like the meaningless string
42 "Tea Time". This is not valid, so we're expecting an error when we
43 submit the form.
44 - Click "Create" to save the event.
45
46Expected result:
47
48 - Form reloads with an error message about the specific mistake.
49 - All field values are retained.
50
51Actual result:
52
53 - Form reloads with an error message about the specific mistake.
54 - Most values are discarded, so I have to re-type the name, description, etc.
55
56---
57
58These steps are **complete** and **self-contained**: anyone can reproduce the
59issue by following these steps. These steps are **clear** and **easy to
60follow**. These steps are also simple and minimal: they don't include any
61extra unnecessary steps.
62
63Finally, these steps explain what the reporter expected to happen, what they
64observed, and how those behaviors differ. This isn't as important when the
65observation is an obvious error like an exception, but can be important if a
66behavior is merely odd or ambiguous.
67
68
69Reliable Reproduction
70=====================
71
72When you file a bug report, the first thing we do to fix it is to try to
73reproduce the problem locally on an up-to-date install of Phorge. We will
74do this by following the steps you provide. If we can recreate the issue
75locally, we can almost always resolve the problem (often very promptly).
76
77However, many reports do not have enough information, are missing important
78steps, or rely on data (like commits, users, other projects, permission
79settings, feed stories, etc) that we don't have access to. We either can't
80follow these steps, or can't reproduce issues by following them.
81
82Reproduction steps must be complete and self-contained, and must allow
83**anyone** to reproduce the issue on a new, empty install of Phorge. If
84the bug you're seeing depends on data or configuration which would not be
85present on a new install, you need to include that information in your steps.
86
87For example, if you're seeing an issue which depends on a particular policy
88setting or configuration setting, you need to include instructions for creating
89the policy or adjusting the setting in your steps.
90
91
92Getting Started
93===============
94
95To generate reproduction steps, first find a sequence of actions which
96reproduce the issue you're seeing reliably.
97
98Next, write down everything you did as clearly as possible. Make sure each step
99is self-contained: anyone should be able to follow your steps, without access
100to private or proprietary data.
101
102Now, to verify that your steps provide a complete, self-contained way to
103reproduce the issue, follow them yourself on a new, empty, up-to-date instance
104of Phorge.
105
106If you can follow your steps and reproduce the issue on a clean instance,
107we'll probably be able to follow them and reproduce the issue ourselves.
108
109If you can't follow your steps because they depend on information which is not
110available on a clean instance (for example, a certain config setting), rewrite
111them to include instructions to create that information (for example, adjusting
112the config to the problematic value).
113
114If you follow your instructions but the issue does not reproduce, the issue
115might already be fixed. Make sure your install is up to date.
116
117If your install is up to date and the issue still doesn't reproduce on a clean
118install, your reproduction steps are missing important information. You need to
119figure out what key element differs between your install and the clean install.
120
121Once you have working reproduction steps, your steps may have details which
122aren't actually necessary to reproduce the issue. You may be able to simplify
123them by removing some steps or describing steps more narrowly. For help, see
124"Simplifying Steps" below.
125
126
127Simplifying Steps
128=================
129
130If you aren't sure how to simplify reproduction steps, this section has some
131advice.
132
133In general, you'll simplify reproduction steps by first finding a scenario
134where the issue reproduces reliably (a "bad" case) and a second, similar
135situation where it does not reproduce (a "good" case). Once you have a "bad"
136case and a "good" case, you'll change the scenarios step-by-step to be more
137similar to each other, until you have two scenarios which differ only a very
138small amount. This remaining difference usually points clearly at the root
139cause of the issue.
140
141For example, suppose you notice that you get an error if you commit a file
142named `A Banana.doc`, but not if you commit a file named `readme.md`. In
143this case, `A Banana.doc` is your "bad" case and `readme.md` is your "good"
144case.
145
146There are several differences between these file names, and any of them might
147be causing the problem. To narrow this down, you can try making the scenarios
148more similar. For example, do these files work?
149
150 - `A_Banana.doc` - Problem with spaces?
151 - `A Banana.md` - File extension issue?
152 - `A Ban.doc` - Path length issue?
153 - `a banana.doc` - Issue with letter case?
154
155Some of these probably work, while others might not. These could lead you to a
156smaller case which reproduces the problem, which might be something like this:
157
158 - Files like `a b`, which contain spaces, do not work.
159 - Files like `a.doc`, which have the `.doc` extension, do not work.
160 - Files like `AAAAAAAAAA`, which have more than 9 letters, do not work.
161 - Files like `A`, which have uppercase letters, do not work.
162
163With a simpler reproduction scenario, you can simplify your steps to be more
164tailored and minimal. This will help us pinpoint the issue more quickly and
165be more certain that we've understood and resolved it.
166
167It is more important that steps be complete than that they be simple, and it's
168acceptable to submit complex instructions if you have difficulty simplifying
169them, so long as they are complete, self-contained, and accurately reproduce
170the issue.
171
172
173Things to Avoid
174===============
175
176These are common mistakes when providing reproduction instructions:
177
178**Insufficient Information**: The most common issue we see is instructions
179which do not have enough information: they are missing critical details which
180are necessary in order to follow them on a clean install.
181
182**Inaccurate Steps**: The second most common issue we see is instructions
183which do not actually reproduce a problem when followed on a clean, up-to-date
184install. Verify your steps by testing them.
185
186**Private Information**: Some users provide reports which hinge on the
187particulars of private commits in proprietary repositories we can not access.
188This is not useful, because we can not examine the underlying commit to figure
189out why it is causing an issue.
190
191Instead, reproduce the issue in a public repository. There are several test
192repositories available which you can push commits to in order to construct a
193reproduction case.
194
195**Screenshots**: Screenshots can be helpful to explain a set of steps or show
196what you're seeing, but they usually aren't sufficient on their own because
197they don't contain all the information we need to reproduce them.
198
199For example, a screenshot may show a particular policy or object, but not have
200enough information for us rebuild a similar object locally.
201
202
203Alternatives
204============
205
206If you have an issue which you can't build reproduction steps for, or which
207only reproduces in your environment, or which you don't want to narrow down
208to a minimal reproduction case, we can't accept it as a bug report. These
209issues are tremendously time consuming for us to pursue and rarely benefit
210more than one install.
211
212
213Next Steps
214==========
215
216Continue by:
217
218 - returning to @{article:Contributing Bug Reports}.