Add a "setup" cache · recaptime.dev/phorge@a88b69a

Add a "setup" cache

Summary:
See T2062. This cache allows us to essentially implement this sort of block:

if (this_code_has_not_run_since_the_last_server_restart()) {
...
}

This will let us do setup checks automatically (i.e., without a specialized setup mode) without imposing hundreds of milliseconds of `git submodule status` and similar checks on every page load, even if an install does not have APC.

Broadly, the major goals here are:

- Reduce user errors and support costs related to misconfiguration (e.g., failure to update submodules).
- Simplify setup and configuration (remove 'phabricator.setup', remove/reduce PHABRICATOR_ENV).
- Move as much configuration to the web as possible (required for SaaS).

Test Plan:
Added this block to webroot/index.php:

$cache = PhabricatorCaches::getSetupCache();
$result = $cache->getKeys(array('x'));
if (empty($result['x'])) {
phlog('Cache miss + set.');
$cache->setKeys(array('x' => 'y'));
} else {
phlog('Cache hit.');
}

Verified it used APC correctly.
Disabled APC and verified it degraded to a reasonable disk-based behavior.

If we miss both of these we end up with no actual caching, but that's the best we can do. This code will also run too early in setup for it to be appropriate to raise exceptions out of this pathway -- later on, we can raise a warning that APC is not installed.

Reviewers: btrahan, vrana

Reviewed By: btrahan

CC: aran

Maniphest Tasks: T2227, T2062

Differential Revision: https://secure.phabricator.com/D4281

epriestley 13 years ago a88b69a4 e05d8d70

+164

2 changed files

expand all

src

__phutil_library_map__.php

applications

cache

PhabricatorCaches.php

src/__phutil_library_map__.php

··· 651 651 'PhabricatorBuiltinPatchList' => 'infrastructure/storage/patch/PhabricatorBuiltinPatchList.php', 652 652 'PhabricatorButtonsExample' => 'applications/uiexample/examples/PhabricatorButtonsExample.php', 653 653 'PhabricatorCacheDAO' => 'applications/cache/storage/PhabricatorCacheDAO.php', 654 + 'PhabricatorCaches' => 'applications/cache/PhabricatorCaches.php', 654 655 'PhabricatorCalendarBrowseController' => 'applications/calendar/controller/PhabricatorCalendarBrowseController.php', 655 656 'PhabricatorCalendarController' => 'applications/calendar/controller/PhabricatorCalendarController.php', 656 657 'PhabricatorCalendarDAO' => 'applications/calendar/storage/PhabricatorCalendarDAO.php',

+163

src/applications/cache/PhabricatorCaches.php

··· 1 + <?php 2 + 3 + /** 4 + * @task setup Setup Cache 5 + */ 6 + final class PhabricatorCaches { 7 + 8 + 9 + /* -( Setup Cache )-------------------------------------------------------- */ 10 + 11 + 12 + /** 13 + * Highly specialized cache for performing setup checks. We use this cache 14 + * to determine if we need to run expensive setup checks (e.g., verifying 15 + * submodule versions, PATH, the presence of binaries, etc.) when the page 16 + * loads. Without it, we would need to run these checks every time. 17 + * 18 + * Normally, this cache is just APC. In the absence of APC, this cache 19 + * degrades into a slow, quirky on-disk cache. 20 + * 21 + * NOTE: Do not use this cache for anything else! It is not a general-purpose 22 + * cache! 23 + * 24 + * @return PhutilKeyValueCacheStack Most qualified available cache stack. 25 + * @task setup 26 + */ 27 + public static function getSetupCache() { 28 + static $cache; 29 + if (!$cache) { 30 + $caches = self::buildSetupCaches(); 31 + $cache = id(new PhutilKeyValueCacheStack()) 32 + ->setCaches($caches) 33 + ->setProfiler(PhutilServiceProfiler::getInstance()); 34 + } 35 + return $cache; 36 + } 37 + 38 + 39 + /** 40 + * @task setup 41 + */ 42 + private static function buildSetupCaches() { 43 + // In most cases, we should have APC. This is an ideal cache for our 44 + // purposes -- it's fast and empties on server restart. 45 + $apc = new PhutilKeyValueCacheAPC(); 46 + if ($apc->isAvailable()) { 47 + return array($apc); 48 + } 49 + 50 + // If we don't have APC, build a poor approximation on disk. This is still 51 + // much better than nothing; some setup steps are quite slow. 52 + $disk_path = self::getSetupCacheDiskCachePath(); 53 + if ($disk_path) { 54 + $disk = new PhutilKeyValueCacheOnDisk(); 55 + $disk->setCacheFile($disk_path); 56 + if ($disk->isAvailable()) { 57 + return array($disk); 58 + } 59 + } 60 + 61 + return array(); 62 + } 63 + 64 + 65 + /** 66 + * @task setup 67 + */ 68 + private static function getSetupCacheDiskCachePath() { 69 + // The difficulty here is in choosing a path which will change on server 70 + // restart (we MUST have this property), but as rarely as possible 71 + // otherwise (we desire this property to give the cache the best hit rate 72 + // we can). 73 + 74 + // In some setups, the parent PID is more stable and longer-lived that the 75 + // PID (e.g., under apache, our PID will be a worker while the ppid will 76 + // be the main httpd process). If we're confident we're running under such 77 + // a setup, we can try to use the PPID as the basis for our cache instead 78 + // of our own PID. 79 + $use_ppid = false; 80 + 81 + switch (php_sapi_name()) { 82 + case 'cli-server': 83 + // This is the PHP5.4+ built-in webserver. We should use the pid 84 + // (the server), not the ppid (probably a shell or something). 85 + $use_ppid = false; 86 + break; 87 + case 'fpm-fcgi': 88 + // We should be safe to use PPID here. 89 + $use_ppid = true; 90 + break; 91 + case 'apache2handler': 92 + // We're definitely safe to use the PPID. 93 + $use_ppid = true; 94 + break; 95 + } 96 + 97 + $pid_basis = getmypid(); 98 + if ($use_ppid) { 99 + if (function_exists('posix_getppid')) { 100 + $parent_pid = posix_getppid(); 101 + // On most systems, normal processes can never have PIDs lower than 100, 102 + // so something likely went wrong if we we get one of these. 103 + if ($parent_pid > 100) { 104 + $pid_basis = $parent_pid; 105 + } 106 + } 107 + } 108 + 109 + // If possible, we also want to know when the process launched, so we can 110 + // drop the cache if a process restarts but gets the same PID an earlier 111 + // process had. "/proc" is not available everywhere (e.g., not on OSX), but 112 + // check if we have it. 113 + $epoch_basis = null; 114 + $stat = @stat("/proc/{$pid_basis}"); 115 + if ($stat !== false) { 116 + $epoch_basis = $stat['ctime']; 117 + } 118 + 119 + $tmp_dir = sys_get_temp_dir(); 120 + 121 + $tmp_path = $tmp_dir.DIRECTORY_SEPARATOR.'phabricator-setup'; 122 + if (!file_exists($tmp_path)) { 123 + @mkdir($tmp_path); 124 + } 125 + 126 + $is_ok = self::testTemporaryDirectory($tmp_path); 127 + if (!$is_ok) { 128 + $tmp_path = $tmp_dir; 129 + $is_ok = self::testTemporaryDirectory($tmp_path); 130 + if (!$is_ok) { 131 + // We can't find anywhere to write the cache, so just bail. 132 + return null; 133 + } 134 + } 135 + 136 + $tmp_name = 'setup-'.$pid_basis; 137 + if ($epoch_basis) { 138 + $tmp_name .= '.'.$epoch_basis; 139 + } 140 + $tmp_name .= '.cache'; 141 + 142 + return $tmp_path.DIRECTORY_SEPARATOR.$tmp_name; 143 + } 144 + 145 + 146 + /** 147 + * @task setup 148 + */ 149 + private static function testTemporaryDirectory($dir) { 150 + if (!@file_exists($dir)) { 151 + return false; 152 + } 153 + if (!@is_dir($dir)) { 154 + return false; 155 + } 156 + if (!@is_writable($dir)) { 157 + return false; 158 + } 159 + 160 + return true; 161 + } 162 + 163 + }

Configure Feed

Configure Feed