recaptime.dev / phorge
@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.) hq.recaptime.dev/wiki/Phorge
phorge phabricator
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Use Lisk sets in fact update iterator

Summary:
Fact engines loading dependent objects are super slow because they load them one by one.
This diff put each page in a Lisk set allowing engines to use `loadRelatives()`.

It also introduces `clearSet()` method which is somewhat neccessary in PHP < 5.3 or with disabled cyclic [[ http://php.net/gc | GC ]].

Test Plan:
$iterator = new PhabricatorFactUpdateIterator(new DifferentialRevision());
foreach ($iterator as $revision) {
$diffs = $revision->loadRelatives(new DifferentialDiff(), 'revisionID');
echo memory_get_usage() . "\n";
}

Experienced not-steadily-increasing memory usage and much faster loading.

Reviewers: epriestley

Reviewed By: epriestley

CC: aran, Korvin

Differential Revision: https://secure.phabricator.com/D3247

vrana f8414915 74b438db

+22 -1
+6 -1
src/applications/fact/extract/PhabricatorFactUpdateIterator.php
··· 28 28 private $position; 29 29 private $ignoreUpdatesDuration = 15; 30 30 31 + private $set; 32 + 31 33 public function __construct(LiskDAO $object) { 32 - $this->object = $object; 34 + $this->set = new LiskDAOSet(); 35 + $this->object = $object->putInSet($this->set); 33 36 $this->position = '0:0'; 34 37 } 35 38 ··· 52 55 53 56 protected function loadPage() { 54 57 list($after_epoch, $after_id) = explode(':', $this->cursor); 58 + 59 + $this->set->clearSet(); 55 60 56 61 // NOTE: We ignore recent updates because once we process an update we'll 57 62 // never process rows behind it again. We need to read only rows which
+16
src/infrastructure/storage/lisk/LiskDAOSet.php
··· 39 39 final class LiskDAOSet { 40 40 private $daos = array(); 41 41 private $relatives = array(); 42 + private $subsets = array(); 42 43 43 44 public function addToSet(LiskDAO $dao) { 44 45 $this->daos[] = $dao; 45 46 $dao->putInSet($this); 46 47 return $this; 47 48 } 49 + 50 + /** 51 + * The main purpose of this method is to break cyclic dependency. 52 + * It removes all objects from this set and all subsets created by it. 53 + */ 54 + final public function clearSet() { 55 + $this->daos = array(); 56 + $this->relatives = array(); 57 + foreach ($this->subsets as $set) { 58 + $set->clearSet(); 59 + } 60 + return $this; 61 + } 62 + 48 63 49 64 /** 50 65 * See @{method:LiskDAO::loadRelatives}. ··· 70 85 $relatives = array(); 71 86 } else { 72 87 $set = new LiskDAOSet(); 88 + $this->subsets[] = $set; 73 89 $relatives = $object->putInSet($set)->loadAllWhere( 74 90 '%C IN (%Ls) %Q', 75 91 $foreign_column,