recaptime.dev / phorge
@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.) hq.recaptime.dev/wiki/Phorge
phorge phabricator
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Bail out of PhabricatorRepositoryGraphCache more aggressively after cache fills

Summary:
Ref PHI109. Ref T11786. We currently test elapsed time every 64 iterations (since iterations are normally very fast), but at least one install is seeing the page timeout after 30 seconds.

One reason could be that cache fills may occur, and are likely to be much slower than normal iterations. In an extreme case, we could do 64 cache fills before checking the time. Tweak thing so that we always check the time after doing a cache fill, regardless of how many iterations have elapsed since the last attempt.

Additionally, this API method currently accepts an arbitrary number of paths, but implicitly limits each cache query to 500ms. If more than 60 paths are passed, this may exceed 30s. Only let the cache churn for a maximum of 10s across all paths.

If this is more the latter issue than the former, this might replace the GraphCache timeouts with `git` timeouts, but at least our understanding of what's going on here will improve.

Test Plan: This is difficult to test convincingly locally, since I can't reproduce the original issue. It still works after these changes, but it worked fine before these changes too.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T11786

Differential Revision: https://secure.phabricator.com/D18692

epriestley 85011a46 17e83b53

+26 -5
+13 -1
src/applications/diffusion/conduit/DiffusionLastModifiedQueryConduitAPIMethod.php
··· 115 115 $graph_cache = new PhabricatorRepositoryGraphCache(); 116 116 117 117 $results = array(); 118 + 119 + // Spend no more than this many total seconds trying to satisfy queries 120 + // via the graph cache. 121 + $remaining_time = 10.0; 118 122 foreach ($map as $path => $commit) { 119 123 $path_id = idx($path_map, $path); 120 124 if (!$path_id) { ··· 125 129 continue; 126 130 } 127 131 132 + $t_start = microtime(true); 128 133 $cache_result = $graph_cache->loadLastModifiedCommitID( 129 134 $commit_id, 130 - $path_id); 135 + $path_id, 136 + $remaining_time); 137 + $t_end = microtime(true); 131 138 132 139 if ($cache_result !== false) { 133 140 $results[$path] = $cache_result; 141 + } 142 + 143 + $remaining_time -= ($t_end - $t_start); 144 + if ($remaining_time <= 0) { 145 + break; 134 146 } 135 147 } 136 148
+13 -4
src/applications/repository/graphcache/PhabricatorRepositoryGraphCache.php
··· 102 102 } 103 103 104 104 // Otherwise, the rebuild gave us the data, so we can keep going. 105 + 106 + $did_fill = true; 107 + } else { 108 + $did_fill = false; 105 109 } 106 110 107 111 // Sanity check so we can survive and recover from bad data. ··· 147 151 $commit_id = $parent_id; 148 152 149 153 // Periodically check if we've spent too long looking for a result 150 - // in the cache, and return so we can fall back to a VCS operation. This 151 - // keeps us from having a degenerate worst case if, e.g., the cache 152 - // is cold and we need to inspect a very large number of blocks 154 + // in the cache, and return so we can fall back to a VCS operation. 155 + // This keeps us from having a degenerate worst case if, e.g., the 156 + // cache is cold and we need to inspect a very large number of blocks 153 157 // to satisfy the query. 154 158 155 - if (((++$iterations) % 64) === 0) { 159 + ++$iterations; 160 + 161 + // If we performed a cache fill in this cycle, always check the time 162 + // limit, since cache fills may take a significant amount of time. 163 + 164 + if ($did_fill || ($iterations % 64 === 0)) { 156 165 $t_end = microtime(true); 157 166 if (($t_end - $t_start) > $time) { 158 167 return false;