recaptime.dev / phorge
@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.) hq.recaptime.dev/wiki/Phorge
phorge phabricator
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

In Git, only use "--find-copies-harder" on small diffs

Summary:
Ref T10423. This flag can cause `git diff` to take an enormously long time (the problem case was a 5M line, 20K file commit).

Instead:

- Run without the flag first.
- If that shows that the diff is definitely small, try again with the flag.
- If that works, return the slower, better output.
- If the fast diff affects too many paths or generating the slow diff takes too long, return the faster, slightly worse output.

The quality of the output differs in how well Git is able to detect "M" and "C" (moves and copies of files).

For example, if you copy `src/` to `srcpro/`, the fast output may not show that you copied files. The slow output will.

I think this is rarely useful for large copies anyway: it's interesting if a 1-2 file diff is a copy, but usually obvious/uninteresting if a 500-file diff is a copy.

Test Plan:
- Ran `bin/repository reparse --change rXnnn` on Git changes.
- Saw fast and slow commands execute normally.
- Tried on a large diff, saw only the fast command execute.

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T10423

Differential Revision: https://secure.phabricator.com/D16266

epriestley d44a5fa9 e2d195e0

+60 -11
+60 -11
src/applications/diffusion/conduit/DiffusionInternalGitRawDiffQueryConduitAPIMethod.php
··· 28 28 protected function getResult(ConduitAPIRequest $request) { 29 29 $drequest = $this->getDiffusionRequest(); 30 30 $repository = $drequest->getRepository(); 31 + $commit = $request->getValue('commit'); 31 32 32 33 if (!$repository->isGit()) { 33 34 throw new Exception( ··· 44 45 list($parents) = $repository->execxLocalCommand( 45 46 'log -n1 --format=%s %s', 46 47 '%P', 47 - $request->getValue('commit')); 48 + $commit); 49 + $use_log = !strlen(trim($parents)); 50 + 51 + // First, get a fast raw diff without "--find-copies-harder". This flag 52 + // produces better results for moves and copies, but is explosively slow 53 + // for large changes to large repositories. See T10423. 54 + $raw = $this->getRawDiff($repository, $commit, $use_log, false); 55 + 56 + // If we got a normal-sized diff (no more than 100 modified files), we'll 57 + // try using "--find-copies-harder" to improve the output. This improved 58 + // output is mostly useful for small changes anyway. 59 + $try_harder = (substr_count($raw, "\n") <= 100); 60 + if ($try_harder) { 61 + try { 62 + $raw = $this->getRawDiff($repository, $commit, $use_log, true); 63 + } catch (Exception $ex) { 64 + // Just ignore any exception we hit, we'll use the fast output 65 + // instead. 66 + } 67 + } 68 + 69 + return $raw; 70 + } 71 + 72 + private function getRawDiff( 73 + PhabricatorRepository $repository, 74 + $commit, 75 + $use_log, 76 + $try_harder) { 77 + 78 + $flags = array( 79 + '-n1', 80 + '-M', 81 + '-C', 82 + '-B', 83 + '--raw', 84 + '-t', 85 + '--abbrev=40', 86 + ); 48 87 49 - $use_log = !strlen(trim($parents)); 88 + if ($try_harder) { 89 + $flags[] = '--find-copies-harder'; 90 + } 91 + 50 92 if ($use_log) { 51 93 // This is the first commit so we need to use "log". We know it's not a 52 94 // merge commit because it couldn't be merging anything, so this is safe. 53 95 54 96 // NOTE: "--pretty=format: " is to disable diff output, we only want the 55 97 // part we get from "--raw". 56 - list($raw) = $repository->execxLocalCommand( 57 - 'log -n1 -M -C -B --find-copies-harder --raw -t '. 58 - '--pretty=format: --abbrev=40 %s', 59 - $request->getValue('commit')); 98 + $future = $repository->getLocalCommandFuture( 99 + 'log %Ls --pretty=format: %s', 100 + $flags, 101 + $commit); 60 102 } else { 61 103 // Otherwise, we can use "diff", which will give us output for merges. 62 104 // We diff against the first parent, as this is generally the expectation 63 105 // and results in sensible behavior. 64 - list($raw) = $repository->execxLocalCommand( 65 - 'diff -n1 -M -C -B --find-copies-harder --raw -t '. 66 - '--abbrev=40 %s^1 %s', 67 - $request->getValue('commit'), 68 - $request->getValue('commit')); 106 + $future = $repository->getLocalCommandFuture( 107 + 'diff %Ls %s^1 %s', 108 + $flags, 109 + $commit, 110 + $commit); 111 + } 112 + 113 + // Don't spend more than 30 seconds generating the slower output. 114 + if ($try_harder) { 115 + $future->setTimeout(30); 69 116 } 117 + 118 + list($raw) = $future->resolvex(); 70 119 71 120 return $raw; 72 121 }