When paging by Ferret "rank", page using "HAVING rank > ...", not "WHERE rank > ..."

Summary:
Ref T13091. The Ferret "rank" column is a function of the query text and looks something like `SELECT ..., 2 + 2 AS rank, ...`.

You can't apply conditions to this kind of dynamic column with a WHERE clause: you get a slightly unhelpful error like "column rank unknown in where clause". You must use HAVING:

```
mysql> SELECT 2 + 2 AS x WHERE x = 4;
ERROR 1054 (42S22): Unknown column 'x' in 'where clause'
mysql> SELECT 2 + 2 AS x HAVING x = 4;
+---+
| x |
+---+
| 4 |
+---+
1 row in set (0.00 sec)
```

Add a flag to paging column definitions to let them specify that they must be applied with HAVING, then apply the whole paging clause with HAVING if any column requires HAVING.

Test Plan:
- In Maniphest, ran a fulltext search matching more than 100 results, ordered by "Relevance", then clicked "Next Page".
- Before patch: query with `... WHERE rank > 123 OR ...` caused MySQL error because `rank` is not a WHERE-able column.
- After patch: query builds as `... HAVING rank > 123 OR ...`, pages properly, no MySQL error.

Reviewers: amckinley

Reviewed By: amckinley

Maniphest Tasks: T13091

Differential Revision: https://secure.phabricator.com/D20298

epriestley 7 years ago 7f905706 f047b90d

+66 -3

1 changed file

expand all

src

infrastructure

query

policy

PhabricatorCursorPagedPolicyAwareQuery.php

+66 -3

src/infrastructure/query/policy/PhabricatorCursorPagedPolicyAwareQuery.php

··· 83 83 84 84 $this->applyExternalCursorConstraintsToQuery($query, $cursor); 85 85 86 + // If we have a Ferret fulltext query, copy it to the subquery so that we 87 + // generate ranking columns appropriately, and compute the correct object 88 + // ranking score for the current query. 89 + if ($this->ferretEngine) { 90 + $query->withFerretConstraint($this->ferretEngine, $this->ferretTokens); 91 + } 92 + 86 93 // We're executing the subquery normally to make sure the viewer can 87 94 // actually see the object, and that it's a completely valid object which 88 95 // passes all filtering and policy checks. You aren't allowed to use an ··· 204 211 get_class($this))); 205 212 } 206 213 214 + if ($this->supportsFerretEngine()) { 215 + if ($this->getFerretTokens()) { 216 + $map += array( 217 + 'rank' => 218 + $cursor->getRawRowProperty(self::FULLTEXT_RANK), 219 + 'fulltext-modified' => 220 + $cursor->getRawRowProperty(self::FULLTEXT_MODIFIED), 221 + 'fulltext-created' => 222 + $cursor->getRawRowProperty(self::FULLTEXT_CREATED), 223 + ); 224 + } 225 + } 226 + 207 227 foreach ($keys as $key) { 208 228 if (!array_key_exists($key, $map)) { 209 229 throw new Exception( ··· 295 315 } 296 316 297 317 protected function didLoadRawRows(array $rows) { 318 + $this->rawCursorRow = last($rows); 319 + 298 320 if ($this->ferretEngine) { 299 321 foreach ($rows as $row) { 300 322 $phid = $row['phid']; ··· 311 333 unset($row[self::FULLTEXT_CREATED]); 312 334 } 313 335 } 314 - 315 - $this->rawCursorRow = last($rows); 316 336 317 337 return $rows; 318 338 } ··· 467 487 */ 468 488 protected function buildWhereClauseParts(AphrontDatabaseConnection $conn) { 469 489 $where = array(); 470 - $where[] = $this->buildPagingClause($conn); 490 + $where[] = $this->buildPagingWhereClause($conn); 471 491 $where[] = $this->buildEdgeLogicWhereClause($conn); 472 492 $where[] = $this->buildSpacesWhereClause($conn); 473 493 $where[] = $this->buildNgramsWhereClause($conn); ··· 482 502 */ 483 503 protected function buildHavingClause(AphrontDatabaseConnection $conn) { 484 504 $having = $this->buildHavingClauseParts($conn); 505 + $having[] = $this->buildPagingHavingClause($conn); 485 506 return $this->formatHavingClause($conn, $having); 486 507 } 487 508 ··· 539 560 /* -( Paging )------------------------------------------------------------- */ 540 561 541 562 563 + private function buildPagingWhereClause(AphrontDatabaseConnection $conn) { 564 + if ($this->shouldPageWithHavingClause()) { 565 + return null; 566 + } 567 + 568 + return $this->buildPagingClause($conn); 569 + } 570 + 571 + private function buildPagingHavingClause(AphrontDatabaseConnection $conn) { 572 + if (!$this->shouldPageWithHavingClause()) { 573 + return null; 574 + } 575 + 576 + return $this->buildPagingClause($conn); 577 + } 578 + 579 + private function shouldPageWithHavingClause() { 580 + // If any of the paging conditions reference dynamic columns, we need to 581 + // put the paging conditions in a "HAVING" clause instead of a "WHERE" 582 + // clause. 583 + 584 + // For example, this happens when paging on the Ferret "rank" column, 585 + // since the "rank" value is computed dynamically in the SELECT statement. 586 + 587 + $orderable = $this->getOrderableColumns(); 588 + $vector = $this->getOrderVector(); 589 + 590 + foreach ($vector as $order) { 591 + $key = $order->getOrderKey(); 592 + $column = $orderable[$key]; 593 + 594 + if (!empty($column['having'])) { 595 + return true; 596 + } 597 + } 598 + 599 + return false; 600 + } 601 + 542 602 /** 543 603 * @task paging 544 604 */ ··· 655 715 'reverse' => 'optional bool', 656 716 'unique' => 'optional bool', 657 717 'null' => 'optional string|null', 718 + 'requires-ferret' => 'optional bool', 719 + 'having' => 'optional bool', 658 720 )); 659 721 } 660 722 ··· 1106 1168 'column' => self::FULLTEXT_RANK, 1107 1169 'type' => 'int', 1108 1170 'requires-ferret' => true, 1171 + 'having' => true, 1109 1172 ); 1110 1173 $columns['fulltext-created'] = array( 1111 1174 'table' => null,

Configure Feed

Configure Feed