recaptime.dev / phorge
@recaptime-dev's working patches + fork for Phorge, a community fork of Phabricator. (Upstream dev and stable branches are at upstream/main and upstream/stable respectively.) hq.recaptime.dev/wiki/Phorge
phorge phabricator
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Restrict maximum number of search tokens per query to 61

Summary:
Currently the Query text field in Search allows an arbitrary number of tokens (characters divided by whitespace) up to a total of 1024 bytes.
If the number of tokens is 62 or higher, this triggers a MariaDB error: `Too many tables; MariaDB can only use 61 tables in a join`

This patch displays a clearer error message via a proper `PhutilSearchQueryCompilerSyntaxException` which can be filtered in server logs.
This only applies to parsing the Query string and not to typeahead fields or such.
(For better understanding, note that a Query term like `foo AND (bar OR meouw)` counts as 5 tokens, not 3 tokens.)

See also rP72cb3d3c84905c0d75074e4ecf74c493e3a2d527 (though that older commit introduced analyzing search tokens while this very change just drops the query with an error, as I personally interpret too many search tokens as PEBKAC).

Closes T15831

Test Plan:
1. Go to `http://phorge.localhost/typeahead/class/PhabricatorSearchDatasource/?__path__=%2ftypeahead%2fclass%2fPhabricatorSearchDatasource%2f&__ajax__=true&__metablock__=569&q=i%20have%20a%20hacker%20and%20scammer%20that%20is%20on%20me%2024%20hoirs%20a%20day%20and%20had%20access%20to%20me%20icloud%20snd%20other%20accounts%20i%20cakled%20everyonevand%20even%20fraud%20cant%20help%20me%20this%20gurl%20is%20smart%20snd%20nadechersrlf%20the%20admin%20of%20my%20accounts%20while%20i%20was%20in%20a%20comma%20in%20the%20hospital%20for%20over%20a%20month%20ina%20half%20she%20is%20rtryingcto%20take%20over%20my%20identity%20and%20i%20refuse%20tovletcthat%20happen%20how%20do%20i%20stop%20her%20from%20being%20the%20admin%20of%20ny%20accounts%20i%20bever%20had%20inevand%20dintvneed%20one%20cause%20she%20doesntvhelp%20me%20she%20hackescinto%20ky%20accounts%20and%20scanms%20mecand&raw=i%20have%20a%20hacker%20and%20scammer%20that%20is%20on%20me%2024%20hoirs%20a%20day%20and%20had%20access%20to%20me%20icloud%20snd%20other%20accounts%20i%20cakled%20everyonevand%20even%20fraud%20cant%20help%20me%20this%20gurl%20is%20smart%20snd%20nadechersrlf%20the%20admin%20of%20my%20accounts%20while%20i%20was%20in%20a%20comma%20in%20the%20hospital%20for%20over%20a%20month%20ina%20half%20she%20is%20rtryingcto%20take%20over%20my%20identity%20and%20i%20refuse%20tovletcthat%20happen%20how%20do%20i%20stop%20her%20from%20being%20the%20admin%20of%20ny%20accounts%20i%20bever%20had%20inevand%20aaa` and do not blow up anymore with `Too many tables; MariaDB can only use 61 tables in a join` when Phorge queries the `user.nametoken` DB table but instead get a more meaningful error message.
2. Go to http://phorge.localhost/search/query/advanced/ and enter something like `ds ad sd rj a fdj` (but with 61 tokens) in the top bar search field, works as expected.
3. Go to http://phorge.localhost/search/query/advanced/ and enter a search string with 62 tokens in the top bar search field, shows error as expected.
4. Enter the same two strings from step 2 and step 3 on http://phorge.localhost in the top bar search field on the right.
5. Go to http://phorge.localhost/search/query/HCvkIIvHK4Db/#R and combine a Query string with 61 tokens with other Typeahead fields (enter some Authors, Tags etc), query still works.
6. Go to an Advanced Search of an application, like http://phorge.localhost/maniphest/query/advanced/, and repeat step 5.
7. Go to http://phorge.localhost/search/query/advanced/ and enter a random CJK string like `㪳㪸㫆㫓㫕㫱` but with 62 characters, see that it does not throw an error, as expected, as `PhutilSearchQueryCompiler` special-cases via `phutil_utf8_is_cjk()`.

Reviewers: O1 Blessed Committers, valerio.bozzolan

Reviewed By: O1 Blessed Committers, valerio.bozzolan

Subscribers: tobiaswiese, valerio.bozzolan, Matthew, Cigaryno

Maniphest Tasks: T15831

Differential Revision: https://we.phorge.it/D26055

Andre Klapper 61ce56ff 97e8d3a1

+21 -1
+21 -1
src/applications/search/compiler/PhutilSearchQueryCompiler.php
··· 52 52 } 53 53 54 54 /** 55 + * @return int 56 + */ 57 + private function getMaxQueryTokens(): int { 58 + return 61; 59 + } 60 + 61 + /** 55 62 * @param array<PhutilSearchQueryToken> $tokens 56 63 * @return string|null 57 64 */ ··· 143 150 if ($query_bytes > $maximum_bytes) { 144 151 throw new PhutilSearchQueryCompilerSyntaxException( 145 152 pht( 146 - 'Query is too long (%s bytes, maximum is %s bytes).', 153 + 'Query is too long (%s bytes, maximum is %s bytes). '. 154 + 'Please use more specific search criteria.', 147 155 new PhutilNumber($query_bytes), 148 156 new PhutilNumber($maximum_bytes))); 149 157 } ··· 297 305 298 306 $tokens[] = $token; 299 307 } 308 + 309 + $query_tokens = count($tokens); 310 + $maximum_tokens = $this->getMaxQueryTokens(); 311 + if ($query_tokens > $maximum_tokens) { 312 + throw new PhutilSearchQueryCompilerSyntaxException( 313 + pht( 314 + 'Query has too many search tokens (%s tokens, maximum is %s '. 315 + 'tokens). Please use more specific search criteria.', 316 + new PhutilNumber($query_tokens), 317 + new PhutilNumber($maximum_tokens))); 318 + } 319 + 300 320 301 321 $results = array(); 302 322 $last_function = null;