Manually prune Git working copy refs instead of using "--prune", to improve "Fetch Refs" behavior

Summary:
Ref T13448. When "Fetch Refs" is configured:

- We switch to a narrow mode when running "ls-remote" against the local working copy. This excludes surplus refs, so we'll incorrectly detect that the local and remote working copies are identical in cases where the local working copy really has surplus refs.
- We rely on "--prune" to remove surplus local refs, but it only prunes refs matching the refspecs we pass "git fetch". Since these refspecs are very narrow under "Fetch Only", the pruning behavior is also very narrow.

Instead:

- When listing local refs, always list everything. If we have too much stuff locally, we want to get rid of it.
- When we identify surplus local refs, explicitly delete them instead of relying on "--prune". We can just do this in all cases so we don't have separate "--prune" and "manual" cases.

Test Plan:
- Created a new repository, observed from a GitHub repository, with many tags/refs/branches. Pulled it.
- Observed lots of refs in `git for-each-ref`.
- Changed "Fetch Refs" to "refs/heads/master".
- Ran `bin/repository pull X --trace --verbose`.

On deciding to do something:

- Before: since "master" did not change, the pull declined to act.
- After: the pull detected surplus refs and deleted them. Since the states then matched, it declined further action.

On pruning:

- Before: if the pull was forced to act, it ran "fetch --prune" with a narrow refspec, which did not prune the working copy.
- After: saw working copy pruned explicitly with "update-ref -d" commands.

Also, set "Fetch Refs" back to the default (empty) and pulled, saw everything pull.

Maniphest Tasks: T13448

Differential Revision: https://secure.phabricator.com/D20892

epriestley 6 years ago 9dbde24d f5f2a0bc

+69 -15

1 changed file

expand all

src

applications

repository

engine

PhabricatorRepositoryPullEngine.php

+69 -15

src/applications/repository/engine/PhabricatorRepositoryPullEngine.php

··· 353 353 // Load the refs we're planning to fetch from the remote repository. 354 354 $remote_refs = $this->loadGitRemoteRefs( 355 355 $repository, 356 - $repository->getRemoteURIEnvelope()); 356 + $repository->getRemoteURIEnvelope(), 357 + $is_local = false); 357 358 358 359 // Load the refs we're planning to fetch from the local repository, by 359 360 // using the local working copy path as the "remote" repository URI. 360 361 $local_refs = $this->loadGitRemoteRefs( 361 362 $repository, 362 - new PhutilOpaqueEnvelope($path)); 363 + new PhutilOpaqueEnvelope($path), 364 + $is_local = true); 365 + 366 + // See T13448. The "git fetch --prune ..." flag only prunes local refs 367 + // matching the refspecs we pass it. If "Fetch Refs" is configured, we'll 368 + // pass it a very narrow list of refspecs, and it won't prune older refs 369 + // that aren't currently subject to fetching. 370 + 371 + // Since we want to prune everything that isn't (a) on the fetch list and 372 + // (b) in the remote, handle pruning of any surplus leftover refs ourselves 373 + // before we fetch anything. 374 + 375 + // (We don't have to do this if "Fetch Refs" isn't set up, since "--prune" 376 + // will work in that case, but it's a little simpler to always go down the 377 + // same code path.) 378 + 379 + $surplus_refs = array(); 380 + foreach ($local_refs as $local_ref => $local_hash) { 381 + $remote_hash = idx($remote_refs, $local_ref); 382 + if ($remote_hash === null) { 383 + $surplus_refs[] = $local_ref; 384 + } 385 + } 386 + 387 + if ($surplus_refs) { 388 + $this->log( 389 + pht( 390 + 'Found %s surplus local ref(s) to delete.', 391 + phutil_count($surplus_refs))); 392 + foreach ($surplus_refs as $surplus_ref) { 393 + $this->log( 394 + pht( 395 + 'Deleting surplus local ref "%s" ("%s").', 396 + $surplus_ref, 397 + $local_refs[$surplus_ref])); 398 + 399 + $repository->execLocalCommand( 400 + 'update-ref -d %R --', 401 + $surplus_ref); 402 + 403 + unset($local_refs[$surplus_ref]); 404 + } 405 + } 363 406 364 407 if ($remote_refs === $local_refs) { 365 408 $this->log( ··· 378 421 // checked out. See T13280. 379 422 380 423 $future = $repository->getRemoteCommandFuture( 381 - 'fetch --prune --update-head-ok -- %P %Ls', 424 + 'fetch --update-head-ok -- %P %Ls', 382 425 $repository->getRemoteURIEnvelope(), 383 426 $fetch_rules); 384 427 ··· 474 517 475 518 private function loadGitRemoteRefs( 476 519 PhabricatorRepository $repository, 477 - PhutilOpaqueEnvelope $remote_envelope) { 520 + PhutilOpaqueEnvelope $remote_envelope, 521 + $is_local) { 478 522 479 - $ref_rules = $this->getGitRefRules($repository); 523 + // See T13448. When listing local remotes, we want to list everything, 524 + // not just refs we expect to fetch. This allows us to detect that we have 525 + // undesirable refs (which have been deleted in the remote, but are still 526 + // present locally) so we can update our state to reflect the correct 527 + // remote state. 480 528 481 - // NOTE: "git ls-remote" does not support "--" until circa January 2016. 482 - // See T12416. None of the flags to "ls-remote" appear dangerous, but 483 - // refuse to list any refs beginning with "-" just in case. 529 + if ($is_local) { 530 + $ref_rules = array(); 531 + } else { 532 + $ref_rules = $this->getGitRefRules($repository); 533 + 534 + // NOTE: "git ls-remote" does not support "--" until circa January 2016. 535 + // See T12416. None of the flags to "ls-remote" appear dangerous, but 536 + // refuse to list any refs beginning with "-" just in case. 484 537 485 - foreach ($ref_rules as $ref_rule) { 486 - if (preg_match('/^-/', $ref_rule)) { 487 - throw new Exception( 488 - pht( 489 - 'Refusing to list potentially dangerous ref ("%s") beginning '. 490 - 'with "-".', 491 - $ref_rule)); 538 + foreach ($ref_rules as $ref_rule) { 539 + if (preg_match('/^-/', $ref_rule)) { 540 + throw new Exception( 541 + pht( 542 + 'Refusing to list potentially dangerous ref ("%s") beginning '. 543 + 'with "-".', 544 + $ref_rule)); 545 + } 492 546 } 493 547 } 494 548

Configure Feed

Configure Feed