@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.) hq.recaptime.dev/wiki/Phorge
phorge phabricator
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Increase the size of the Diffusion commit cache

Summary:
Ref T12296. This cache is used to cache Git ref heads (branches, tags, etc). Reasonable repositories may have more than 2048 of these.

When we miss the cache, we need to single-get refs to check them, which is relatively expensive.

Increasing the size of the cache to 65535 should only require about 7.5MB of RAM.

Additionally, fill only as much of the cache as actually fits. The FIFO nature of the cache can get us into trouble otherwise.

If we insert "A, B, C, D" and then lookup A, B, C, D, but the cache has maximum size 3, we get this:

- Insert A, B, C, D: cache is now "B, C, D".
- Lookup A: miss, single get, insert, purge, cache is now "C, D, A".
- Lookup B: miss, singel get, insert, purge, cache is now "D, A, B".

Test Plan:
- Reduced cache size to 5, observed reasonable behavior on the `array_slice()` locally with `bin/repository update` + `var_dump()`.
- Used this script to estimate the size of 65535 cache entries as 7.5MB:

```
epriestley@orbital ~ $ cat size.php
<?php

$cache = array();

$mem_start = memory_get_usage();
for ($ii = 0; $ii < 65535; $ii++) {
$cache[sha1($ii)] = true;
}

echo number_format(memory_get_usage() - $mem_start)." bytes\n";
epriestley@orbital ~ $ php -f size.php
7,602,176 bytes
```

Reviewers: chad

Reviewed By: chad

Maniphest Tasks: T12296

Differential Revision: https://secure.phabricator.com/D17409

+11 -2
+11 -2
src/applications/repository/engine/PhabricatorRepositoryDiscoveryEngine.php
··· 14 14 private $commitCache = array(); 15 15 private $workingSet = array(); 16 16 17 - const MAX_COMMIT_CACHE_SIZE = 2048; 17 + const MAX_COMMIT_CACHE_SIZE = 65535; 18 18 19 19 20 20 /* -( Discovering Repositories )------------------------------------------- */ ··· 476 476 return; 477 477 } 478 478 479 + $max_size = self::MAX_COMMIT_CACHE_SIZE; 480 + 481 + // If we're filling more identifiers than would fit in the cache, ignore 482 + // the ones that don't fit. Because the cache is FIFO, overfilling it can 483 + // cause the entire cache to miss. See T12296. 484 + if (count($identifiers) > $max_size) { 485 + $identifiers = array_slice($identifiers, 0, $max_size); 486 + } 487 + 479 488 // When filling the cache we ignore commits which have been marked as 480 489 // unreachable, treating them as though they do not exist. When recording 481 490 // commits later we'll revive commits that exist but are unreachable. ··· 492 501 $this->commitCache[$commit->getCommitIdentifier()] = true; 493 502 } 494 503 495 - while (count($this->commitCache) > self::MAX_COMMIT_CACHE_SIZE) { 504 + while (count($this->commitCache) > $max_size) { 496 505 array_shift($this->commitCache); 497 506 } 498 507 }