@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.)
hq.recaptime.dev/wiki/Phorge
phorge
phabricator
When migrating files between storage engines with "bin/files migrate ...", skip expired temporary files
Summary:
See T7148. This just cheats us out of a weird sort of race where we:
- Dump an instance, including some `F123` which is a temporary file which expires in 3 minutes.
- A few minutes later, the daemons delete the data for that file.
- A few minutes after that, we try to `bin/files migrate --copy` to copy the data from S3 into the MySQL blob store.
- This fails since the data is already gone.
Instead, just skip these files since they're already dead to us.
Test Plan: Faked this locally, will migrate the PHI769 instance on `aux001`.
Reviewers: amckinley
Reviewed By: amckinley
Differential Revision: https://secure.phabricator.com/D19536
+26
+26
src/applications/files/management/PhabricatorFilesManagementMigrateWorkflow.php
+26
src/applications/files/management/PhabricatorFilesManagementMigrateWorkflow.php
···
85
85
foreach ($iterator as $file) {
86
86
$monogram = $file->getMonogram();
87
87
88
+
// See T7148. When we export data for an instance, we copy all the data
89
+
// for Files from S3 into the database dump so that the database dump is
90
+
// a complete, standalone archive of all the data. In the general case,
91
+
// installs may have a similar process using "--copy" to create a more
92
+
// complete backup.
93
+
94
+
// When doing this, we may run into temporary files which have been
95
+
// deleted between the time we took the original dump and the current
96
+
// timestamp. These files can't be copied since the data no longer
97
+
// exists: the daemons on the live install already deleted it.
98
+
99
+
// Simply avoid this whole mess by declining to migrate expired temporary
100
+
// files. They're as good as dead anyway.
101
+
102
+
$ttl = $file->getTTL();
103
+
if ($ttl) {
104
+
if ($ttl < PhabricatorTime::getNow()) {
105
+
echo tsprintf(
106
+
"%s\n",
107
+
pht(
108
+
'%s: Skipping expired temporary file.',
109
+
$monogram));
110
+
continue;
111
+
}
112
+
}
113
+
88
114
$engine_key = $file->getStorageEngine();
89
115
$engine = idx($engines, $engine_key);
90
116