+144 -67

docs/specs.md

reviewed

··· 143 143 144 144 The mapping from Skyreader subscription to Glean subscription: 145 145 146 146 - | Skyreader field | Glean field | Notes | 147 147 - | --------------- | ------------- | ------------------------------------ | 148 148 - | `feedUrl` | `feed_url` | Direct mapping | 149 149 - | `title` | `title` | Direct mapping | 150 150 - | `siteUrl` | `site_url` | Stored on the feed record | 151 151 - | `createdAt` | `added_at` | Direct mapping | 152 152 - | _(none)_ | `category` | Empty (Skyreader has no categories) | 146 146 + | Skyreader field | Glean field | Notes | 147 147 + | --------------- | ----------- | ----------------------------------- | 148 148 + | `feedUrl` | `feed_url` | Direct mapping | 149 149 + | `title` | `title` | Direct mapping | 150 150 + | `siteUrl` | `site_url` | Stored on the feed record | 151 151 + | `createdAt` | `added_at` | Direct mapping | 152 152 + | _(none)_ | `category` | Empty (Skyreader has no categories) | 153 153 154 154 If a Glean subscription already exists for the same `feed_url`, the existing one is kept. If the existing subscription has no URI (was created locally without PDS sync), the Skyreader URI/CID is backfilled. 155 155 ··· 661 661 ); 662 662 ``` 663 663 664 664 - Glean has two complementary recommendation signals: 664 664 + Glean uses a multi-signal recommendation system that combines subscription overlap, like patterns, social graph distance, and user behavior feedback. 665 665 + 666 666 + ### 7.1 Signals 665 667 666 666 - - **Subscriptions** (Jaccard similarity): "Who reads the same feeds?" → feed and people discovery 667 667 - - **Likes** (co-occurrence): "Who likes the same articles?" → article and feed discovery 668 668 + | Signal | Source | Weight (default) | Description | 669 669 + |--------|--------|-------------------|-------------| 670 670 + | Subscription | `subscriptions` | 1.0 | Jaccard over subscriber sets between similar users | 671 671 + | Like | `likes` | 0.5 | Time-decayed like co-occurrence (30-day half-life) | 672 672 + | Tag | `annotations.tags` | 0.3 | Jaccard over annotation tag sets | 673 673 + | Social | `follow_distances` | 0.7 | Follow distance: 1-hop=1.0, 2-hop=0.3 | 674 674 + | Popularity | `feeds.subscriber_count` | 0.2 | `log(1 + subscribers) / log(1 + max)` | 675 675 + | Category | `subscriptions.category` | 0.4 | Boost feeds matching user's existing categories | 668 676 669 669 - ### 7.1 Feed Co-occurrence (Jaccard Similarity) 677 677 + ### 7.2 Feed Co-occurrence (Jaccard Similarity) 670 678 671 679 For any two feeds, the similarity is the Jaccard index of their subscriber sets: 672 680 ··· 674 682 J(A, B) = |subscribers(A) ∩ subscribers(B)| / |subscribers(A) ∪ subscribers(B)| 675 683 ``` 676 684 677 677 - This is recomputed periodically (cron job) or incrementally when subscriptions change. 685 685 + Feed description text similarity is also computed (word overlap after stopword removal) and added as a boost. 678 686 679 679 - ### 7.2 User Similarity 687 687 + ### 7.3 User Similarity 680 688 681 681 - For any two users, compute Jaccard over their subscription sets: 689 689 + For any two users, compute Jaccard over their subscription sets, plus like co-occurrence (time-decayed) and tag overlap: 682 690 683 691 ``` 684 684 - J(U1, U2) = |feeds(U1) ∩ feeds(U2)| / |feeds(U1) ∪ feeds(U2)| 692 692 + J(U1, U2) = jaccard_subscriptions + 0.3 * jaccard_likes + 0.2 * jaccard_tags + follow_boost 685 693 ``` 686 694 687 687 - ### 7.3 Recommendation Algorithms 695 695 + Like overlap uses exponential time decay: `EXP(-0.023 * age_days)` (30-day half-life). 688 696 689 689 - **Feed recommendations (on glean.at):** 697 697 + ### 7.4 On-Demand Scoring 698 698 + 699 699 + Recommendations are computed **on-demand** at query time, not pre-materialized. This avoids write amplification on every cron run. 690 700 691 691 - 1. Find users with Jaccard > 0.2 (similar readers) 692 692 - 2. Collect feeds those users subscribe to that the target user does not 693 693 - 3. Rank by frequency (how many similar users subscribe) and average similarity 694 694 - 4. Return top N feeds as recommendations 701 701 + **Feed recommendation score** (computed in SQL): 695 702 696 703 ``` 697 697 - score(feed) = Σ J(target, U) for each user U subscribed to feed 704 704 + score = sub_signal * w_sub 705 705 + + like_signal * w_like 706 706 + + social_signal * w_social 707 707 + + pop_signal * w_pop 708 708 + + category_signal * w_category 698 709 ``` 699 710 700 700 - **Article recommendations (on glean.at, from likes):** 711 711 + Where: 712 712 + - `sub_signal = SUM(jaccard(target, U))` for similar users U subscribed to feed 713 713 + - `like_signal = SUM(jaccard(target, U) * time_decay)` for likes in that feed by similar users 714 714 + - `social_signal = SUM(distance_weight)` from follow_distances 715 715 + - `pop_signal = log(1 + subscriber_count) / log(1 + max_subscribers)` 716 716 + - `category_signal = 1` if feed description matches user's top categories 701 717 702 702 - 1. Find users who liked articles that the target user also liked 703 703 - 2. Collect articles those users liked that the target has not 704 704 - 3. Rank by frequency and recency 705 705 - 4. Return top N articles as recommendations 718 718 + **Article recommendation score**: 706 719 707 720 ``` 708 708 - score(article) = Σ J(target, U) for each similar user U who liked the article 721 721 + score = like_signal * w_like 722 722 + + social_signal * w_social 723 723 + + recency_signal * 0.2 709 724 ``` 710 725 711 711 - **People recommendations (to follow on Bluesky):** 726 726 + ### 7.5 User Feedback (Dismiss) 727 727 + 728 728 + Users can dismiss recommendations they don't want to see again: 729 729 + 730 730 + - `POST /feeds/dismiss` — dismiss a feed recommendation 731 731 + - `POST /articles/dismiss` — dismiss an article recommendation 732 732 + - Dismissals are stored locally in `dismissed_recommendations` (not on PDS) 733 733 + - Dismissed items are excluded from all future recommendation queries 734 734 + - Auto-dismiss: items shown >15 times over >30 days without action are auto-dismissed 712 735 713 713 - 1. Compute user similarity for all pairs 714 714 - 2. Return users with highest Jaccard, linking to their Bluesky profile for follow 736 736 + Impression tracking (`recommendation_impressions`) records how many times each recommendation was shown and whether the user acted on it. 715 737 716 716 - ### 7.4 Implementation 738 738 + ### 7.6 Auto-Tuned Signal Weights 717 739 718 718 - For the initial version, brute-force Jaccard with SQLite is sufficient (scale: ~10k users, ~100k subscriptions). The query is: 740 740 + Each user has a row in `user_signal_weights` with per-signal weights. When a user acts on a recommendation (subscribes, likes), the dominant signal that produced that recommendation is rewarded: 719 741 720 720 - ```sql 721 721 - SELECT s2.feed_url, COUNT(*) as overlap_count 722 722 - FROM subscriptions s1 723 723 - JOIN subscriptions s2 ON s1.feed_url = s2.feed_url 724 724 - WHERE s1.user_did = ? AND s2.user_did != ? 725 725 - AND s2.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) 726 726 - GROUP BY s2.feed_url 727 727 - ORDER BY overlap_count DESC 728 728 - LIMIT 20; 742 742 + ``` 743 743 + new_weight = MAX(0.1, MIN(3.0, old_weight * (1 + learning_rate * delta))) 729 744 ``` 730 745 731 731 - For larger scale, move to MinHash + LSH (banded hashing) to approximate Jaccard in sub-linear time. 746 746 + - `learning_rate = 0.1`, `delta = +1` for reward, `-1` for penalty 747 747 + - Only activates after `minActionsTune = 5` positive actions 748 748 + - Defaults are used when no row exists for a user 749 749 + 750 750 + ### 7.7 Social Graph 751 751 + 752 752 + Follow distances (1-hop and 2-hop) are pre-computed in `follow_distances` during the cron job: 753 753 + 754 754 + - 1-hop: direct follows (weight 1.0) 755 755 + - 2-hop: friends-of-friends (weight 0.3) 756 756 + - 3-hop is excluded due to noise and computational cost 757 757 + 758 758 + ### 7.8 Diversity & Freshness 759 759 + 760 760 + After scoring, diversity filtering is applied in Go (not SQL): 761 761 + 762 762 + - **Domain diversity**: max 2 feeds from the same domain in results 763 763 + - **Category diversity**: max 3 feeds from the same category in results 764 764 + - This prevents recommendation clustering on a single source 765 765 + 766 766 + ### 7.9 Cold Start 767 767 + 768 768 + New users with <5 subscriptions get a fallback strategy: 769 769 + 770 770 + 1. Feeds from 1-hop followed users (70% weight) 771 771 + 2. Globally popular feeds by subscriber count (30% weight) 732 772 733 733 - ### 7.5 Clustering Engine (Cron) 773 773 + ### 7.10 Clustering Engine (Cron) 734 774 735 775 A background goroutine runs on a configurable schedule (`GLEAN_CLUSTER_INTERVAL`, default 10m): 736 776 737 737 - 1. **Compute feed similarity**: Batch-update the `feed_similarity` table (Jaccard over subscriber sets) 738 738 - 2. **Compute user similarity**: Batch-update the `user_similarity` table (Jaccard over subscription sets, boosted by follow relationships) 739 739 - 3. **Generate feed recommendations**: Materialize top feed recommendations per user into `user_feed_recommendations` 740 740 - 4. **Generate article recommendations**: Materialize top article recommendations per user into `user_article_recommendations` 777 777 + 1. **Compute feed similarity**: Batch-update `feed_similarity` table (Jaccard over subscriber sets + description similarity) 778 778 + 2. **Compute user similarity**: Batch-update `user_similarity` table (subscription Jaccard + time-decayed likes + tags + follow boost) 779 779 + 3. **Compute follow distances**: 1-hop and 2-hop from `follows` table 780 780 + 4. **Compute signal profiles**: Per-user category/tag/like summaries 781 781 + 5. **Auto-dismiss stale**: Dismiss items shown >15 times over >30 days without action 741 782 742 783 Jetstream ingestion and record indexing happen in a separate persistent goroutine (the Jetstream consumer), not in the cron. 743 784 785 785 + ### 7.11 New Database Tables 786 786 + 744 787 ```sql 745 745 - CREATE TABLE user_feed_recommendations ( 746 746 - user_did TEXT NOT NULL REFERENCES users(did), 747 747 - feed_url TEXT NOT NULL REFERENCES feeds(feed_url), 748 748 - score REAL NOT NULL, 749 749 - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, 750 750 - PRIMARY KEY (user_did, feed_url) 788 788 + CREATE TABLE dismissed_recommendations ( 789 789 + user_did TEXT NOT NULL REFERENCES users(did), 790 790 + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), 791 791 + target_id TEXT NOT NULL, 792 792 + reason TEXT, 793 793 + dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, 794 794 + PRIMARY KEY (user_did, target_type, target_id) 751 795 ); 752 796 753 753 - CREATE TABLE user_article_recommendations ( 754 754 - user_did TEXT NOT NULL REFERENCES users(did), 755 755 - feed_url TEXT NOT NULL, 756 756 - article_url TEXT NOT NULL, 757 757 - score REAL NOT NULL, 758 758 - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, 759 759 - PRIMARY KEY (user_did, feed_url, article_url) 797 797 + CREATE TABLE recommendation_impressions ( 798 798 + user_did TEXT NOT NULL REFERENCES users(did), 799 799 + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), 800 800 + target_id TEXT NOT NULL, 801 801 + first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, 802 802 + last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, 803 803 + shown_count INTEGER NOT NULL DEFAULT 1, 804 804 + acted BOOLEAN NOT NULL DEFAULT 0, 805 805 + PRIMARY KEY (user_did, target_type, target_id) 806 806 + ); 807 807 + 808 808 + CREATE TABLE follow_distances ( 809 809 + user_a TEXT NOT NULL, 810 810 + user_b TEXT NOT NULL, 811 811 + distance INTEGER NOT NULL CHECK(distance IN (1, 2)), 812 812 + PRIMARY KEY (user_a, user_b) 813 813 + ); 814 814 + 815 815 + CREATE TABLE user_signal_weights ( 816 816 + user_did TEXT PRIMARY KEY REFERENCES users(did), 817 817 + w_sub REAL NOT NULL DEFAULT 1.0, 818 818 + w_like REAL NOT NULL DEFAULT 0.5, 819 819 + w_tag REAL NOT NULL DEFAULT 0.3, 820 820 + w_social REAL NOT NULL DEFAULT 0.7, 821 821 + w_pop REAL NOT NULL DEFAULT 0.2, 822 822 + w_category REAL NOT NULL DEFAULT 0.4, 823 823 + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP 824 824 + ); 825 825 + 826 826 + CREATE TABLE user_signal_profiles ( 827 827 + user_did TEXT PRIMARY KEY REFERENCES users(did), 828 828 + total_likes INTEGER NOT NULL DEFAULT 0, 829 829 + total_tags INTEGER NOT NULL DEFAULT 0, 830 830 + top_categories TEXT, 831 831 + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP 760 832 ); 761 833 ``` 762 834 ··· 778 850 | `/feeds/add` | POST | Add a single feed URL | 779 851 | `/feeds/remove` | DELETE | Remove a feed | 780 852 | `/feeds/refresh` | POST | Refresh all subscribed feeds | 781 781 - | `/feeds/clear` | POST | Clear all subscriptions | 853 853 + | `/feeds/clear` | POST | Clear all subscriptions | 854 854 + | `/feeds/dismiss` | POST | Dismiss a feed recommendation | 782 855 | `/articles` | GET | Read articles (paginated, filterable by feed) | 783 856 | `/articles/{id}` | GET | Article detail view | 784 857 | `/articles/{id}/read` | POST | Mark article as read | 785 858 | `/articles/{id}/unread` | POST | Mark article as unread | 786 859 | `/articles/{id}/like` | POST | Like an article | 787 860 | `/articles/{id}/fetch-content` | POST | Fetch full article content from original URL | 788 788 - | `/articles/mark-all-read` | POST | Mark all articles as read | 789 789 - | `/trending` | GET | Community feed: articles ranked by likes | 861 861 + | `/articles/mark-all-read` | POST | Mark all articles as read | 862 862 + | `/articles/dismiss` | POST | Dismiss an article recommendation | 863 863 + | `/trending` | GET | Community feed: articles ranked by likes | 790 864 | `/library` | GET | Liked articles and annotations | 791 865 | `/library/create` | POST | Create annotation on an article | 792 866 | `/library/{id}/delete` | POST | Delete an annotation | ··· 846 920 │ │ └── metrics.go # Prometheus metrics definitions 847 921 │ ├── cluster/ 848 922 │ │ ├── jaccard.go # Jaccard similarity computation 849 849 - │ │ ├── recommender.go # Feed + people recommendation queries 923 923 + │ │ ├── recommender.go # Feed + people recommendation queries (on-demand) 924 924 + │ │ ├── scoring.go # Multi-signal composite scoring queries 925 925 + │ │ ├── social.go # Follow-distance computation (1-2 hop) 926 926 + │ │ ├── dismiss.go # Dismiss + impression tracking 927 927 + │ │ ├── weights.go # Bandit-style signal weight auto-tuning 928 928 + │ │ ├── diversity.go # Post-query domain/category diversity filtering 850 929 │ │ └── cron.go # Background recomputation scheduler 851 930 │ ├── server/ 852 931 │ │ ├── server.go # HTTP server, router setup ··· 993 1072 994 1073 ## 13. Future Considerations 995 1074 996 996 - - **MinHash/LSH**: Replace brute-force Jaccard when user count exceeds ~50k 997 997 - - **Full-text search**: Add FTS5 virtual table on articles for search 998 1075 - **Email digest**: Periodic email with top articles from subscribed feeds

+26 -54

internal/atproto/xrpc.go

reviewed

··· 8 8 "strings" 9 9 10 10 "github.com/go-chi/chi/v5" 11 11 + 12 12 + "pkg.rbrt.fr/glean/internal/cluster" 11 13 ) 12 14 13 15 type XRPCHandler struct { 14 14 - db *sql.DB 16 16 + db *sql.DB 17 17 + engine *cluster.Engine 15 18 } 16 19 17 17 - func NewXRPCHandler(db *sql.DB) *XRPCHandler { 18 18 - return &XRPCHandler{db: db} 20 20 + func NewXRPCHandler(db *sql.DB, engine *cluster.Engine) *XRPCHandler { 21 21 + return &XRPCHandler{db: db, engine: engine} 19 22 } 20 23 21 24 func (h *XRPCHandler) ListSubscriptions(w http.ResponseWriter, r *http.Request) { ··· 303 306 repo := r.URL.Query().Get("repo") 304 307 limit := min(parseIntParam(r, "limit", 20), 50) 305 308 306 306 - feedRows, err := h.db.QueryContext(r.Context(), ` 307 307 - SELECT r.feed_url, f.title, f.site_url, f.description, f.subscriber_count, r.score 308 308 - FROM user_feed_recommendations r 309 309 - JOIN feeds f ON r.feed_url = f.feed_url 310 310 - WHERE r.user_did = ? 311 311 - ORDER BY r.score DESC 312 312 - LIMIT ? 313 313 - `, repo, limit) 309 309 + ctx := r.Context() 310 310 + 311 311 + feedRecs, err := h.engine.GetFeedRecommendations(ctx, repo, limit) 314 312 if err != nil { 315 313 http.Error(w, err.Error(), http.StatusInternalServerError) 316 314 return 317 315 } 318 318 - defer feedRows.Close() 319 316 320 320 - feeds := make([]RecommendedFeed, 0) 321 321 - for feedRows.Next() { 322 322 - var feedURL, title, siteURL, description string 323 323 - var subscriberCount int 324 324 - var score float64 325 325 - if err := feedRows.Scan(&feedURL, &title, &siteURL, &description, &subscriberCount, &score); err != nil { 326 326 - http.Error(w, err.Error(), http.StatusInternalServerError) 327 327 - return 328 328 - } 317 317 + feeds := make([]RecommendedFeed, 0, len(feedRecs)) 318 318 + for _, rec := range feedRecs { 329 319 feeds = append(feeds, RecommendedFeed{ 330 330 - FeedURL: feedURL, 331 331 - Title: title, 332 332 - SiteURL: siteURL, 333 333 - Description: description, 334 334 - SubscriberCount: subscriberCount, 335 335 - Score: score, 320 320 + FeedURL: rec.FeedURL, 321 321 + Title: rec.Title, 322 322 + SiteURL: rec.SiteURL, 323 323 + Description: rec.Description, 324 324 + SubscriberCount: rec.SubscriberCount, 325 325 + Score: rec.Score, 336 326 }) 337 327 } 338 328 339 339 - peopleRows, err := h.db.QueryContext(r.Context(), ` 340 340 - SELECT u.did, u.handle, u.display_name, u.avatar_url, s.jaccard, s.common_feeds 341 341 - FROM user_similarity s 342 342 - JOIN users u ON ( 343 343 - CASE WHEN s.user_a = ? THEN s.user_b ELSE s.user_a END 344 344 - ) = u.did 345 345 - WHERE s.user_a = ? OR s.user_b = ? 346 346 - ORDER BY s.jaccard DESC 347 347 - LIMIT ? 348 348 - `, repo, repo, repo, limit) 329 329 + peopleRecs, err := h.engine.GetPeopleRecommendations(ctx, repo, limit) 349 330 if err != nil { 350 331 http.Error(w, err.Error(), http.StatusInternalServerError) 351 332 return 352 333 } 353 353 - defer peopleRows.Close() 354 334 355 355 - people := make([]RecommendedPerson, 0) 356 356 - for peopleRows.Next() { 357 357 - var did, handle string 358 358 - var displayName, avatar sql.NullString 359 359 - var jaccard float64 360 360 - var commonFeeds int 361 361 - if err := peopleRows.Scan(&did, &handle, &displayName, &avatar, &jaccard, &commonFeeds); err != nil { 362 362 - http.Error(w, err.Error(), http.StatusInternalServerError) 363 363 - return 364 364 - } 335 335 + people := make([]RecommendedPerson, 0, len(peopleRecs)) 336 336 + for _, rec := range peopleRecs { 365 337 people = append(people, RecommendedPerson{ 366 366 - DID: did, 367 367 - Handle: handle, 368 368 - DisplayName: displayName.String, 369 369 - Avatar: avatar.String, 370 370 - Jaccard: jaccard, 371 371 - CommonFeeds: commonFeeds, 338 338 + DID: rec.DID, 339 339 + Handle: rec.Handle, 340 340 + DisplayName: rec.DisplayName, 341 341 + Avatar: rec.AvatarURL, 342 342 + Jaccard: rec.Jaccard, 343 343 + CommonFeeds: rec.CommonFeeds, 372 344 }) 373 345 } 374 346

+8 -2

internal/cluster/cron.go

reviewed

··· 32 32 if err := c.engine.ComputeUserSimilarity(ctx); err != nil { 33 33 c.engine.logger.Error("user similarity failed", "error", err) 34 34 } 35 35 - if err := c.engine.ComputeRecommendations(ctx); err != nil { 36 36 - c.engine.logger.Error("recommendations failed", "error", err) 35 35 + if err := c.engine.ComputeFollowDistances(ctx); err != nil { 36 36 + c.engine.logger.Error("follow distances failed", "error", err) 37 37 + } 38 38 + if err := c.engine.ComputeSignalProfiles(ctx); err != nil { 39 39 + c.engine.logger.Error("signal profiles failed", "error", err) 40 40 + } 41 41 + if err := c.engine.AutoDismissStale(ctx, 15, 30); err != nil { 42 42 + c.engine.logger.Error("auto dismiss failed", "error", err) 37 43 } 38 44 c.engine.mu.Unlock() 39 45 }

+82

internal/cluster/dismiss.go

reviewed

··· 1 1 + package cluster 2 2 + 3 3 + import ( 4 4 + "context" 5 5 + "time" 6 6 + ) 7 7 + 8 8 + type Impression struct { 9 9 + TargetType string 10 10 + TargetID string 11 11 + } 12 12 + 13 13 + func (e *Engine) DismissFeed(ctx context.Context, userDID, feedURL, reason string) error { 14 14 + _, err := e.db.ExecContext(ctx, ` 15 15 + INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason) 16 16 + VALUES (?, 'feed', ?, ?) 17 17 + ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP 18 18 + `, userDID, feedURL, reason) 19 19 + return err 20 20 + } 21 21 + 22 22 + func (e *Engine) DismissArticle(ctx context.Context, userDID, articleURL, reason string) error { 23 23 + _, err := e.db.ExecContext(ctx, ` 24 24 + INSERT INTO dismissed_recommendations (user_did, target_type, target_id, reason) 25 25 + VALUES (?, 'article', ?, ?) 26 26 + ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET reason = excluded.reason, dismissed_at = CURRENT_TIMESTAMP 27 27 + `, userDID, articleURL, reason) 28 28 + return err 29 29 + } 30 30 + 31 31 + func (e *Engine) RecordImpressions(ctx context.Context, userDID string, impressions []Impression) error { 32 32 + tx, err := e.db.BeginTx(ctx, nil) 33 33 + if err != nil { 34 34 + return err 35 35 + } 36 36 + defer func() { _ = tx.Rollback() }() 37 37 + 38 38 + for _, imp := range impressions { 39 39 + _, err := tx.ExecContext(ctx, ` 40 40 + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count) 41 41 + VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, 1) 42 42 + ON CONFLICT(user_did, target_type, target_id) DO UPDATE SET 43 43 + last_shown_at = CURRENT_TIMESTAMP, 44 44 + shown_count = shown_count + 1 45 45 + `, userDID, imp.TargetType, imp.TargetID) 46 46 + if err != nil { 47 47 + return err 48 48 + } 49 49 + } 50 50 + return tx.Commit() 51 51 + } 52 52 + 53 53 + func (e *Engine) MarkImpressionActed(ctx context.Context, userDID, targetType, targetID string) error { 54 54 + _, err := e.db.ExecContext(ctx, ` 55 55 + UPDATE recommendation_impressions SET acted = 1 56 56 + WHERE user_did = ? AND target_type = ? AND target_id = ? 57 57 + `, userDID, targetType, targetID) 58 58 + return err 59 59 + } 60 60 + 61 61 + func (e *Engine) AutoDismissStale(ctx context.Context, minShownCount int, maxAgeDays int) error { 62 62 + cutoff := time.Now().AddDate(0, 0, -maxAgeDays).Format(time.RFC3339) 63 63 + 64 64 + _, err := e.db.ExecContext(ctx, ` 65 65 + INSERT OR IGNORE INTO dismissed_recommendations (user_did, target_type, target_id, reason, dismissed_at) 66 66 + SELECT user_did, target_type, target_id, 'auto_stale', CURRENT_TIMESTAMP 67 67 + FROM recommendation_impressions 68 68 + WHERE acted = 0 69 69 + AND shown_count >= ? 70 70 + AND first_shown_at < ? 71 71 + `, minShownCount, cutoff) 72 72 + return err 73 73 + } 74 74 + 75 75 + func (e *Engine) IsFeedDismissed(ctx context.Context, userDID, feedURL string) (bool, error) { 76 76 + var count int 77 77 + err := e.db.QueryRowContext(ctx, ` 78 78 + SELECT COUNT(1) FROM dismissed_recommendations 79 79 + WHERE user_did = ? AND target_type = 'feed' AND target_id = ? 80 80 + `, userDID, feedURL).Scan(&count) 81 81 + return count > 0, err 82 82 + }

+68

internal/cluster/diversity.go

reviewed

··· 1 1 + package cluster 2 2 + 3 3 + import ( 4 4 + "net/url" 5 5 + "strings" 6 6 + ) 7 7 + 8 8 + const maxPerDomain = 2 9 9 + const maxPerCategory = 3 10 10 + 11 11 + func ApplyDiversity(candidates []*FeedRecommendation, topN int) []*FeedRecommendation { 12 12 + domainCount := make(map[string]int, len(candidates)) 13 13 + categoryCount := make(map[string]int) 14 14 + result := make([]*FeedRecommendation, 0, topN) 15 15 + 16 16 + for _, c := range candidates { 17 17 + if len(result) >= topN { 18 18 + break 19 19 + } 20 20 + 21 21 + domain := extractDomain(c.SiteURL) 22 22 + if domain != "" && domainCount[domain] >= maxPerDomain { 23 23 + continue 24 24 + } 25 25 + 26 26 + cat := extractCategory(c.Description) 27 27 + if cat != "" && categoryCount[cat] >= maxPerCategory { 28 28 + continue 29 29 + } 30 30 + 31 31 + if domain != "" { 32 32 + domainCount[domain]++ 33 33 + } 34 34 + if cat != "" { 35 35 + categoryCount[cat]++ 36 36 + } 37 37 + result = append(result, c) 38 38 + } 39 39 + 40 40 + return result 41 41 + } 42 42 + 43 43 + func extractDomain(siteURL string) string { 44 44 + if siteURL == "" { 45 45 + return "" 46 46 + } 47 47 + u, err := url.Parse(siteURL) 48 48 + if err != nil { 49 49 + return "" 50 50 + } 51 51 + host := u.Hostname() 52 52 + parts := strings.Split(host, ".") 53 53 + if len(parts) > 2 { 54 54 + return strings.Join(parts[len(parts)-2:], ".") 55 55 + } 56 56 + return host 57 57 + } 58 58 + 59 59 + func extractCategory(description string) string { 60 60 + if description == "" { 61 61 + return "" 62 62 + } 63 63 + words := strings.Fields(strings.ToLower(description)) 64 64 + if len(words) == 0 { 65 65 + return "" 66 66 + } 67 67 + return words[0] 68 68 + }

+14 -99

internal/cluster/jaccard.go

reviewed

··· 9 9 ) 10 10 11 11 type Config struct { 12 12 - SimilarityThreshold float64 13 13 - FollowBoost float64 14 14 - LikesWeight float64 15 15 - TagsWeight float64 16 16 - DescriptionWeight float64 12 12 + FollowBoost float64 13 13 + LikesWeight float64 14 14 + TagsWeight float64 15 15 + DescriptionWeight float64 17 16 } 18 17 19 18 func DefaultConfig() Config { 20 19 return Config{ 21 21 - SimilarityThreshold: 0.2, 22 22 - FollowBoost: 0.5, 23 23 - LikesWeight: 0.3, 24 24 - TagsWeight: 0.2, 25 25 - DescriptionWeight: 0.15, 20 20 + FollowBoost: 0.5, 21 21 + LikesWeight: 0.3, 22 22 + TagsWeight: 0.2, 23 23 + DescriptionWeight: 0.15, 26 24 } 27 25 } 28 26 ··· 37 35 return &Engine{db: db, logger: logger, config: DefaultConfig()} 38 36 } 39 37 40 40 - func (e *Engine) ComputeArticleRecommendations(ctx context.Context) error { 41 41 - tx, err := e.db.BeginTx(ctx, nil) 42 42 - if err != nil { 43 43 - return err 44 44 - } 45 45 - defer func() { _ = tx.Rollback() }() 46 46 - 47 47 - if _, err := tx.ExecContext(ctx, `DELETE FROM user_article_recommendations`); err != nil { 48 48 - return err 49 49 - } 50 50 - 51 51 - query := fmt.Sprintf(` 52 52 - INSERT INTO user_article_recommendations (user_did, feed_url, article_url, score) 53 53 - SELECT targets.target, l.feed_url, l.article_url, SUM(targets.jaccard) AS score 54 54 - FROM ( 55 55 - SELECT us.user_a AS target, us.user_b AS peer, us.jaccard 56 56 - FROM user_similarity us 57 57 - WHERE us.jaccard > %g 58 58 - UNION ALL 59 59 - SELECT us.user_b AS target, us.user_a AS peer, us.jaccard 60 60 - FROM user_similarity us 61 61 - WHERE us.jaccard > %g 62 62 - ) targets 63 63 - JOIN likes l ON l.author_did = targets.peer 64 64 - WHERE NOT EXISTS ( 65 65 - SELECT 1 FROM subscriptions sub WHERE sub.user_did = targets.target AND sub.feed_url = l.feed_url 66 66 - ) 67 67 - AND NOT EXISTS ( 68 68 - SELECT 1 FROM likes ul WHERE ul.author_did = targets.target AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url 69 69 - ) 70 70 - GROUP BY targets.target, l.feed_url, l.article_url 71 71 - ORDER BY score DESC 72 72 - `, e.config.SimilarityThreshold, e.config.SimilarityThreshold) 73 73 - 74 74 - if _, err := tx.ExecContext(ctx, query); err != nil { 75 75 - return err 76 76 - } 77 77 - 78 78 - e.logger.Info("article recommendations computed") 79 79 - return tx.Commit() 80 80 - } 81 81 - 82 38 func (e *Engine) ComputeFeedSimilarity(ctx context.Context) error { 83 39 tx, err := e.db.BeginTx(ctx, nil) 84 40 if err != nil { ··· 260 216 } 261 217 if _, err := tx.ExecContext(ctx, ` 262 218 INSERT INTO _likes_overlap (user_a, user_b, common) 263 263 - SELECT l1.author_did, l2.author_did, COUNT(*) 219 219 + SELECT l1.author_did, l2.author_did, 220 220 + CAST(SUM( 221 221 + EXP(-0.023 * CAST(julianday('now') - julianday(l1.created_at) AS REAL)) 222 222 + * EXP(-0.023 * CAST(julianday('now') - julianday(l2.created_at) AS REAL)) 223 223 + ) AS INTEGER) 264 224 FROM likes l1 265 225 JOIN likes l2 ON l1.feed_url = l2.feed_url AND l1.article_url = l2.article_url 266 226 AND l1.author_did < l2.author_did 227 227 + WHERE l1.created_at IS NOT NULL AND l2.created_at IS NOT NULL 267 228 GROUP BY l1.author_did, l2.author_did 268 229 `); err != nil { 269 230 return err ··· 400 361 e.logger.Info("user similarity computed") 401 362 return tx.Commit() 402 363 } 403 403 - 404 404 - func (e *Engine) ComputeRecommendations(ctx context.Context) error { 405 405 - tx, err := e.db.BeginTx(ctx, nil) 406 406 - if err != nil { 407 407 - return err 408 408 - } 409 409 - defer func() { _ = tx.Rollback() }() 410 410 - 411 411 - if _, err := tx.ExecContext(ctx, `DELETE FROM user_feed_recommendations`); err != nil { 412 412 - return err 413 413 - } 414 414 - 415 415 - recQuery := fmt.Sprintf(` 416 416 - INSERT INTO user_feed_recommendations (user_did, feed_url, score) 417 417 - SELECT target, feed_url, SUM(jaccard) AS score 418 418 - FROM ( 419 419 - SELECT us.user_a AS target, s.feed_url, us.jaccard 420 420 - FROM user_similarity us 421 421 - JOIN subscriptions s ON s.user_did = us.user_b 422 422 - WHERE us.jaccard > %g 423 423 - AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = us.user_a) 424 424 - 425 425 - UNION ALL 426 426 - 427 427 - SELECT us.user_b AS target, s.feed_url, us.jaccard 428 428 - FROM user_similarity us 429 429 - JOIN subscriptions s ON s.user_did = us.user_a 430 430 - WHERE us.jaccard > %g 431 431 - AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = us.user_b) 432 432 - ) 433 433 - GROUP BY target, feed_url 434 434 - ORDER BY score DESC 435 435 - `, e.config.SimilarityThreshold, e.config.SimilarityThreshold) 436 436 - 437 437 - if _, err := tx.ExecContext(ctx, recQuery); err != nil { 438 438 - return err 439 439 - } 440 440 - 441 441 - e.logger.Info("feed recommendations computed") 442 442 - 443 443 - if err := tx.Commit(); err != nil { 444 444 - return err 445 445 - } 446 446 - 447 447 - return e.ComputeArticleRecommendations(ctx) 448 448 - }

+323 -36

internal/cluster/jaccard_test.go

reviewed

··· 2 2 3 3 import ( 4 4 "context" 5 5 + "fmt" 5 6 "os" 6 7 "testing" 7 8 ··· 64 65 } 65 66 } 66 67 68 68 + func seedFollowData(t *testing.T, ctx context.Context, database *db.DB) { 69 69 + t.Helper() 70 70 + follows := []struct{ user, target string }{ 71 71 + {"did:test:alice", "did:test:bob"}, 72 72 + {"did:test:bob", "did:test:carol"}, 73 73 + } 74 74 + for _, f := range follows { 75 75 + _, err := database.ExecContext(ctx, `INSERT OR IGNORE INTO follows (user_did, target_did) VALUES (?, ?)`, f.user, f.target) 76 76 + assert.NilError(t, err) 77 77 + } 78 78 + } 79 79 + 67 80 func TestComputeFeedSimilarity(t *testing.T) { 68 81 ctx := context.Background() 69 82 database := setupClusterTestDB(t) ··· 94 107 assert.Assert(t, count > 0, "expected user similarity pairs") 95 108 } 96 109 97 97 - func TestComputeRecommendations_GeneratesFeedRecsForNewUser(t *testing.T) { 110 110 + func TestOnDemandFeedRecommendations(t *testing.T) { 98 111 ctx := context.Background() 99 112 database := setupClusterTestDB(t) 100 113 seedClusterData(t, ctx, database) ··· 102 115 engine := NewEngine(database.DB, slog.Default()) 103 116 assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) 104 117 assert.NilError(t, engine.ComputeUserSimilarity(ctx)) 105 105 - assert.NilError(t, engine.ComputeRecommendations(ctx)) 106 118 107 119 recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10) 108 120 assert.NilError(t, err) ··· 117 129 assert.Assert(t, found, "carol should be recommended feeds she doesn't subscribe to") 118 130 } 119 131 120 120 - func TestComputeRecommendations_NoSelfRecommendations(t *testing.T) { 132 132 + func TestNoSelfRecommendations(t *testing.T) { 121 133 ctx := context.Background() 122 134 database := setupClusterTestDB(t) 123 135 seedClusterData(t, ctx, database) ··· 125 137 engine := NewEngine(database.DB, slog.Default()) 126 138 assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) 127 139 assert.NilError(t, engine.ComputeUserSimilarity(ctx)) 128 128 - assert.NilError(t, engine.ComputeRecommendations(ctx)) 129 140 130 141 recs, err := engine.GetFeedRecommendations(ctx, "did:test:alice", 10) 131 142 assert.NilError(t, err) ··· 141 152 } 142 153 } 143 154 144 144 - func TestLikesBasedSimilarity(t *testing.T) { 155 155 + func TestDismissedFeedsExcluded(t *testing.T) { 145 156 ctx := context.Background() 146 157 database := setupClusterTestDB(t) 147 158 seedClusterData(t, ctx, database) 148 159 149 149 - _, err := database.ExecContext(ctx, `INSERT INTO articles (feed_url, guid, title, url) VALUES (?, ?, ?, ?)`, 150 150 - "https://a.com/feed", "art1", "Article 1", "https://a.com/art1") 160 160 + engine := NewEngine(database.DB, slog.Default()) 161 161 + assert.NilError(t, engine.ComputeFeedSimilarity(ctx)) 162 162 + assert.NilError(t, engine.ComputeUserSimilarity(ctx)) 163 163 + 164 164 + assert.NilError(t, engine.DismissFeed(ctx, "did:test:carol", "https://a.com/feed", "not_interested")) 165 165 + 166 166 + recs, err := engine.GetFeedRecommendations(ctx, "did:test:carol", 10) 151 167 assert.NilError(t, err) 152 152 - _, err = database.ExecContext(ctx, `INSERT INTO articles (feed_url, guid, title, url) VALUES (?, ?, ?, ?)`, 153 153 - "https://a.com/feed", "art2", "Article 2", "https://a.com/art2") 168 168 + 169 169 + for _, r := range recs { 170 170 + assert.Assert(t, r.FeedURL != "https://a.com/feed", 171 171 + "dismissed feed should not appear in recommendations") 172 172 + } 173 173 + } 174 174 + 175 175 + func TestIsFeedDismissed(t *testing.T) { 176 176 + ctx := context.Background() 177 177 + database := setupClusterTestDB(t) 178 178 + seedClusterData(t, ctx, database) 179 179 + 180 180 + engine := NewEngine(database.DB, slog.Default()) 181 181 + 182 182 + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://a.com/feed") 154 183 assert.NilError(t, err) 184 184 + assert.Assert(t, !dismissed, "feed should not be dismissed initially") 155 185 156 156 - _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`, 157 157 - "at://alice/like/1", "did:test:alice", "https://a.com/feed", "https://a.com/art1") 186 186 + assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "not_interested")) 187 187 + 188 188 + dismissed, err = engine.IsFeedDismissed(ctx, "did:test:alice", "https://a.com/feed") 158 189 assert.NilError(t, err) 159 159 - _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`, 160 160 - "at://alice/like/2", "did:test:alice", "https://a.com/feed", "https://a.com/art2") 190 190 + assert.Assert(t, dismissed, "feed should be dismissed after dismiss call") 191 191 + } 192 192 + 193 193 + func TestRecordImpressions(t *testing.T) { 194 194 + ctx := context.Background() 195 195 + database := setupClusterTestDB(t) 196 196 + seedClusterData(t, ctx, database) 197 197 + 198 198 + engine := NewEngine(database.DB, slog.Default()) 199 199 + 200 200 + impressions := []Impression{ 201 201 + {TargetType: "feed", TargetID: "https://a.com/feed"}, 202 202 + {TargetType: "feed", TargetID: "https://b.com/feed"}, 203 203 + } 204 204 + assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions)) 205 205 + 206 206 + var count int 207 207 + assert.NilError(t, database.QueryRowContext(ctx, 208 208 + `SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = 'did:test:alice'`).Scan(&count)) 209 209 + assert.Equal(t, count, 2) 210 210 + 211 211 + assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions)) 212 212 + 213 213 + var shownCount int 214 214 + assert.NilError(t, database.QueryRowContext(ctx, 215 215 + `SELECT shown_count FROM recommendation_impressions WHERE user_did = 'did:test:alice' AND target_id = 'https://a.com/feed'`).Scan(&shownCount)) 216 216 + assert.Equal(t, shownCount, 2, "shown_count should increment on repeated impression") 217 217 + } 218 218 + 219 219 + func TestMarkImpressionActed(t *testing.T) { 220 220 + ctx := context.Background() 221 221 + database := setupClusterTestDB(t) 222 222 + seedClusterData(t, ctx, database) 223 223 + 224 224 + engine := NewEngine(database.DB, slog.Default()) 225 225 + 226 226 + impressions := []Impression{{TargetType: "feed", TargetID: "https://a.com/feed"}} 227 227 + assert.NilError(t, engine.RecordImpressions(ctx, "did:test:alice", impressions)) 228 228 + 229 229 + assert.NilError(t, engine.MarkImpressionActed(ctx, "did:test:alice", "feed", "https://a.com/feed")) 230 230 + 231 231 + var acted bool 232 232 + assert.NilError(t, database.QueryRowContext(ctx, 233 233 + `SELECT acted FROM recommendation_impressions WHERE user_did = 'did:test:alice' AND target_id = 'https://a.com/feed'`).Scan(&acted)) 234 234 + assert.Assert(t, acted, "impression should be marked as acted") 235 235 + } 236 236 + 237 237 + func TestComputeFollowDistances(t *testing.T) { 238 238 + ctx := context.Background() 239 239 + database := setupClusterTestDB(t) 240 240 + seedClusterData(t, ctx, database) 241 241 + seedFollowData(t, ctx, database) 242 242 + 243 243 + engine := NewEngine(database.DB, slog.Default()) 244 244 + assert.NilError(t, engine.ComputeFollowDistances(ctx)) 245 245 + 246 246 + var d1, d2 int 247 247 + assert.NilError(t, database.QueryRowContext(ctx, 248 248 + `SELECT COUNT(*) FROM follow_distances WHERE distance = 1`).Scan(&d1)) 249 249 + assert.NilError(t, database.QueryRowContext(ctx, 250 250 + `SELECT COUNT(*) FROM follow_distances WHERE distance = 2`).Scan(&d2)) 251 251 + assert.Assert(t, d1 >= 2, "expected at least 2 direct follow distances") 252 252 + assert.Assert(t, d2 >= 1, "expected at least 1 two-hop distance (alice -> bob -> carol)") 253 253 + 254 254 + var dist int 255 255 + assert.NilError(t, database.QueryRowContext(ctx, 256 256 + `SELECT distance FROM follow_distances WHERE user_a = 'did:test:alice' AND user_b = 'did:test:carol'`).Scan(&dist)) 257 257 + assert.Equal(t, dist, 2, "alice should be 2 hops from carol") 258 258 + } 259 259 + 260 260 + func TestAutoDismissStale(t *testing.T) { 261 261 + ctx := context.Background() 262 262 + database := setupClusterTestDB(t) 263 263 + seedClusterData(t, ctx, database) 264 264 + 265 265 + engine := NewEngine(database.DB, slog.Default()) 266 266 + 267 267 + _, err := database.ExecContext(ctx, ` 268 268 + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) 269 269 + VALUES ('did:test:alice', 'feed', 'https://stale.com/feed', datetime('now', '-31 days'), datetime('now'), 20, 0) 270 270 + `) 161 271 assert.NilError(t, err) 162 162 - _, err = database.ExecContext(ctx, `INSERT INTO likes (uri, author_did, feed_url, article_url, created_at) VALUES (?, ?, ?, ?, datetime('now'))`, 163 163 - "at://carol/like/1", "did:test:carol", "https://a.com/feed", "https://a.com/art1") 272 272 + 273 273 + assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30)) 274 274 + 275 275 + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://stale.com/feed") 276 276 + assert.NilError(t, err) 277 277 + assert.Assert(t, dismissed, "stale recommendation should be auto-dismissed") 278 278 + } 279 279 + 280 280 + func TestAutoDismissStale_DoesNotDismissRecent(t *testing.T) { 281 281 + ctx := context.Background() 282 282 + database := setupClusterTestDB(t) 283 283 + seedClusterData(t, ctx, database) 284 284 + 285 285 + engine := NewEngine(database.DB, slog.Default()) 286 286 + 287 287 + _, err := database.ExecContext(ctx, ` 288 288 + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) 289 289 + VALUES ('did:test:alice', 'feed', 'https://recent.com/feed', datetime('now'), datetime('now'), 5, 0) 290 290 + `) 291 291 + assert.NilError(t, err) 292 292 + 293 293 + assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30)) 294 294 + 295 295 + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://recent.com/feed") 296 296 + assert.NilError(t, err) 297 297 + assert.Assert(t, !dismissed, "recent impression should not be auto-dismissed") 298 298 + } 299 299 + 300 300 + func TestAutoDismissStale_DoesNotDismissActed(t *testing.T) { 301 301 + ctx := context.Background() 302 302 + database := setupClusterTestDB(t) 303 303 + seedClusterData(t, ctx, database) 304 304 + 305 305 + engine := NewEngine(database.DB, slog.Default()) 306 306 + 307 307 + _, err := database.ExecContext(ctx, ` 308 308 + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) 309 309 + VALUES ('did:test:alice', 'feed', 'https://acted.com/feed', datetime('now', '-31 days'), datetime('now'), 20, 1) 310 310 + `) 311 311 + assert.NilError(t, err) 312 312 + 313 313 + assert.NilError(t, engine.AutoDismissStale(ctx, 15, 30)) 314 314 + 315 315 + dismissed, err := engine.IsFeedDismissed(ctx, "did:test:alice", "https://acted.com/feed") 164 316 assert.NilError(t, err) 317 317 + assert.Assert(t, !dismissed, "acted recommendation should not be auto-dismissed") 318 318 + } 319 319 + 320 320 + func TestDiversityFiltering(t *testing.T) { 321 321 + candidates := []*FeedRecommendation{ 322 322 + {FeedURL: "https://a.com/1", SiteURL: "https://a.com", Score: 1.0}, 323 323 + {FeedURL: "https://a.com/2", SiteURL: "https://a.com", Score: 0.9}, 324 324 + {FeedURL: "https://a.com/3", SiteURL: "https://a.com", Score: 0.8}, 325 325 + {FeedURL: "https://b.com/1", SiteURL: "https://b.com", Score: 0.7}, 326 326 + {FeedURL: "https://b.com/2", SiteURL: "https://b.com", Score: 0.6}, 327 327 + {FeedURL: "https://c.com/1", SiteURL: "https://c.com", Score: 0.5}, 328 328 + } 329 329 + 330 330 + result := ApplyDiversity(candidates, 6) 331 331 + 332 332 + aCount := 0 333 333 + bCount := 0 334 334 + cCount := 0 335 335 + for _, r := range result { 336 336 + switch extractDomain(r.SiteURL) { 337 337 + case "a.com": 338 338 + aCount++ 339 339 + case "b.com": 340 340 + bCount++ 341 341 + case "c.com": 342 342 + cCount++ 343 343 + } 344 344 + } 345 345 + assert.Assert(t, aCount <= maxPerDomain, "should limit feeds from same domain") 346 346 + assert.Assert(t, len(result) <= 6, "should respect topN limit") 347 347 + assert.Assert(t, cCount >= 1, "should include feeds from different domains") 348 348 + } 349 349 + 350 350 + func TestDiversityFiltering_EmptySiteURL(t *testing.T) { 351 351 + candidates := []*FeedRecommendation{ 352 352 + {FeedURL: "https://a.com/1", SiteURL: "", Score: 1.0}, 353 353 + {FeedURL: "https://b.com/1", SiteURL: "", Score: 0.9}, 354 354 + } 355 355 + result := ApplyDiversity(candidates, 5) 356 356 + assert.Equal(t, len(result), 2, "feeds without site_url should not be filtered out") 357 357 + } 358 358 + 359 359 + func TestSignalWeights_Default(t *testing.T) { 360 360 + ctx := context.Background() 361 361 + database := setupClusterTestDB(t) 362 362 + seedClusterData(t, ctx, database) 165 363 166 364 engine := NewEngine(database.DB, slog.Default()) 167 167 - assert.NilError(t, engine.ComputeUserSimilarity(ctx)) 365 365 + w := engine.GetWeights(ctx, "did:test:alice") 366 366 + 367 367 + assert.Equal(t, w.WSub, 1.0) 368 368 + assert.Equal(t, w.WLike, 0.5) 369 369 + assert.Equal(t, w.WTag, 0.3) 370 370 + assert.Equal(t, w.WSocial, 0.7) 371 371 + assert.Equal(t, w.WPop, 0.2) 372 372 + assert.Equal(t, w.WCategory, 0.4) 373 373 + } 374 374 + 375 375 + func TestSignalWeights_RewardPenalize(t *testing.T) { 376 376 + ctx := context.Background() 377 377 + database := setupClusterTestDB(t) 378 378 + seedClusterData(t, ctx, database) 379 379 + 380 380 + engine := NewEngine(database.DB, slog.Default()) 381 381 + 382 382 + _, err := database.ExecContext(ctx, ` 383 383 + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) 384 384 + VALUES ('did:test:alice', 'feed', 'https://a.com/feed', datetime('now'), datetime('now'), 1, 1) 385 385 + `) 386 386 + assert.NilError(t, err) 387 387 + for i := range minActionsTune { 388 388 + _, err = database.ExecContext(ctx, ` 389 389 + INSERT INTO recommendation_impressions (user_did, target_type, target_id, first_shown_at, last_shown_at, shown_count, acted) 390 390 + VALUES ('did:test:alice', 'feed', ?, datetime('now'), datetime('now'), 1, 1) 391 391 + `, fmt.Sprintf("https://%d.com/feed", i)) 392 392 + assert.NilError(t, err) 393 393 + } 168 394 169 169 - var jaccard float64 170 170 - var commonLikes int 171 171 - assert.NilError(t, database.QueryRowContext(ctx, 172 172 - `SELECT jaccard, common_likes FROM user_similarity WHERE user_a = ? AND user_b = ?`, 173 173 - "did:test:alice", "did:test:carol").Scan(&jaccard, &commonLikes)) 174 174 - assert.Equal(t, commonLikes, 1, "alice and carol share 1 liked article") 175 175 - assert.Assert(t, jaccard > 0, "likes should contribute to similarity, got %f", jaccard) 395 395 + engine.RewardSignal(ctx, "did:test:alice", "social") 396 396 + 397 397 + w := engine.GetWeights(ctx, "did:test:alice") 398 398 + assert.Assert(t, w.WSocial > 0.7, "rewarding social signal should increase w_social, got %f", w.WSocial) 176 399 } 177 400 178 178 - func TestTagsBasedSimilarity(t *testing.T) { 401 401 + func TestColdStartRecommendations(t *testing.T) { 179 402 ctx := context.Background() 180 403 database := setupClusterTestDB(t) 181 404 seedClusterData(t, ctx, database) 405 405 + seedFollowData(t, ctx, database) 182 406 183 183 - _, err := database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`, 184 184 - "at://alice/ann/1", "did:test:alice", "https://a.com/feed", "https://a.com/art1", "go,programming") 407 407 + engine := NewEngine(database.DB, slog.Default()) 408 408 + assert.NilError(t, engine.ComputeFollowDistances(ctx)) 409 409 + 410 410 + _, err := database.ExecContext(ctx, `UPDATE feeds SET subscriber_count = 2 WHERE feed_url = 'https://a.com/feed'`) 411 411 + assert.NilError(t, err) 412 412 + _, err = database.ExecContext(ctx, `UPDATE feeds SET subscriber_count = 2 WHERE feed_url = 'https://b.com/feed'`) 413 413 + assert.NilError(t, err) 414 414 + 415 415 + _, err = database.ExecContext(ctx, `INSERT INTO users (did, handle) VALUES (?, ?)`, "did:test:newuser", "newuser") 185 416 assert.NilError(t, err) 186 186 - _, err = database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`, 187 187 - "at://alice/ann/2", "did:test:alice", "https://a.com/feed", "https://a.com/art2", "rust,programming") 417 417 + 418 418 + recs, err := engine.ColdStartRecommendations(ctx, "did:test:newuser", 10) 188 419 assert.NilError(t, err) 189 189 - _, err = database.ExecContext(ctx, `INSERT INTO annotations (uri, author_did, feed_url, article_url, tags, created_at) VALUES (?, ?, ?, ?, ?, datetime('now'))`, 190 190 - "at://carol/ann/1", "did:test:carol", "https://c.com/feed", "https://c.com/art1", "go,web") 420 420 + assert.Assert(t, len(recs) > 0, "new user should get cold start recommendations") 421 421 + } 422 422 + 423 423 + func TestColdStartRecommendations_NotTriggeredForEstablishedUser(t *testing.T) { 424 424 + ctx := context.Background() 425 425 + database := setupClusterTestDB(t) 426 426 + seedClusterData(t, ctx, database) 427 427 + 428 428 + engine := NewEngine(database.DB, slog.Default()) 429 429 + 430 430 + recs, err := engine.ColdStartRecommendations(ctx, "did:test:alice", 10) 191 431 assert.NilError(t, err) 432 432 + assert.Assert(t, recs == nil, "established user should not get cold start recommendations") 433 433 + } 434 434 + 435 435 + func TestOnDemandPeopleRecommendations(t *testing.T) { 436 436 + ctx := context.Background() 437 437 + database := setupClusterTestDB(t) 438 438 + seedClusterData(t, ctx, database) 192 439 193 440 engine := NewEngine(database.DB, slog.Default()) 194 441 assert.NilError(t, engine.ComputeUserSimilarity(ctx)) 195 442 196 196 - var jaccard float64 197 197 - var commonTags int 443 443 + recs, err := engine.GetPeopleRecommendations(ctx, "did:test:carol", 10) 444 444 + assert.NilError(t, err) 445 445 + assert.Assert(t, len(recs) > 0, "carol should get people recommendations") 446 446 + } 447 447 + 448 448 + func TestDismissArticle(t *testing.T) { 449 449 + ctx := context.Background() 450 450 + database := setupClusterTestDB(t) 451 451 + seedClusterData(t, ctx, database) 452 452 + 453 453 + engine := NewEngine(database.DB, slog.Default()) 454 454 + 455 455 + assert.NilError(t, engine.DismissArticle(ctx, "did:test:alice", "https://a.com/article1", "not_interested")) 456 456 + 457 457 + var count int 198 458 assert.NilError(t, database.QueryRowContext(ctx, 199 199 - `SELECT jaccard, common_tags FROM user_similarity WHERE user_a = ? AND user_b = ?`, 200 200 - "did:test:alice", "did:test:carol").Scan(&jaccard, &commonTags)) 201 201 - assert.Equal(t, commonTags, 1, "alice and carol share 1 tag (go)") 202 202 - assert.Assert(t, jaccard > 0, "tags should contribute to similarity, got %f", jaccard) 459 459 + `SELECT COUNT(*) FROM dismissed_recommendations WHERE user_did = 'did:test:alice' AND target_type = 'article'`).Scan(&count)) 460 460 + assert.Equal(t, count, 1) 461 461 + } 462 462 + 463 463 + func TestComputeSignalProfiles(t *testing.T) { 464 464 + ctx := context.Background() 465 465 + database := setupClusterTestDB(t) 466 466 + seedClusterData(t, ctx, database) 467 467 + 468 468 + engine := NewEngine(database.DB, slog.Default()) 469 469 + assert.NilError(t, engine.ComputeSignalProfiles(ctx)) 470 470 + 471 471 + var count int 472 472 + assert.NilError(t, database.QueryRowContext(ctx, `SELECT COUNT(*) FROM user_signal_profiles`).Scan(&count)) 473 473 + assert.Assert(t, count >= 3, "expected signal profiles for all users") 474 474 + } 475 475 + 476 476 + func TestDismissFeed_Idempotent(t *testing.T) { 477 477 + ctx := context.Background() 478 478 + database := setupClusterTestDB(t) 479 479 + seedClusterData(t, ctx, database) 480 480 + 481 481 + engine := NewEngine(database.DB, slog.Default()) 482 482 + 483 483 + assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "reason1")) 484 484 + assert.NilError(t, engine.DismissFeed(ctx, "did:test:alice", "https://a.com/feed", "reason2")) 485 485 + 486 486 + var count int 487 487 + assert.NilError(t, database.QueryRowContext(ctx, 488 488 + `SELECT COUNT(*) FROM dismissed_recommendations WHERE user_did = 'did:test:alice' AND target_type = 'feed'`).Scan(&count)) 489 489 + assert.Equal(t, count, 1, "duplicate dismiss should not create extra rows") 203 490 } 204 491 205 492 func TestDescriptionBasedFeedSimilarity(t *testing.T) {

-165

internal/cluster/recommender.go

reviewed

··· 1 1 - package cluster 2 2 - 3 3 - import ( 4 4 - "context" 5 5 - "database/sql" 6 6 - ) 7 7 - 8 8 - type FeedRecommendation struct { 9 9 - FeedURL string 10 10 - Title string 11 11 - SiteURL string 12 12 - Description string 13 13 - SubscriberCount int 14 14 - FaviconURL string 15 15 - Score float64 16 16 - } 17 17 - 18 18 - type PersonRecommendation struct { 19 19 - DID string 20 20 - Handle string 21 21 - DisplayName string 22 22 - AvatarURL string 23 23 - Jaccard float64 24 24 - CommonFeeds int 25 25 - CommonLikes int 26 26 - CommonTags int 27 27 - } 28 28 - 29 29 - type ArticleRecommendation struct { 30 30 - ArticleID int64 31 31 - Title string 32 32 - URL string 33 33 - FeedURL string 34 34 - FeedTitle string 35 35 - Author string 36 36 - Summary string 37 37 - Published sql.NullTime 38 38 - Score float64 39 39 - } 40 40 - 41 41 - type SimilarFeed struct { 42 42 - FeedURL string 43 43 - Title string 44 44 - SiteURL string 45 45 - Description string 46 46 - FeedType string 47 47 - Jaccard float64 48 48 - } 49 49 - 50 50 - func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { 51 51 - rows, err := e.db.QueryContext(ctx, ` 52 52 - SELECT r.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''), 53 53 - COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''), r.score 54 54 - FROM user_feed_recommendations r 55 55 - JOIN feeds f ON f.feed_url = r.feed_url 56 56 - WHERE r.user_did = ? 57 57 - AND r.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) 58 58 - ORDER BY r.score DESC 59 59 - LIMIT ? 60 60 - `, userDID, userDID, limit) 61 61 - if err != nil { 62 62 - return nil, err 63 63 - } 64 64 - defer rows.Close() 65 65 - 66 66 - var results []*FeedRecommendation 67 67 - for rows.Next() { 68 68 - rec := &FeedRecommendation{} 69 69 - if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, 70 70 - &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil { 71 71 - return nil, err 72 72 - } 73 73 - results = append(results, rec) 74 74 - } 75 75 - return results, rows.Err() 76 76 - } 77 77 - 78 78 - func (e *Engine) GetPeopleRecommendations(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) { 79 79 - rows, err := e.db.QueryContext(ctx, ` 80 80 - SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''), 81 81 - sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0) 82 82 - FROM ( 83 83 - SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ? 84 84 - UNION ALL 85 85 - SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ? 86 86 - ) sim 87 87 - JOIN users u ON u.did = sim.peer_did 88 88 - WHERE u.handle IS NOT NULL AND u.handle != '' 89 89 - ORDER BY sim.jaccard DESC 90 90 - LIMIT ? 91 91 - `, userDID, userDID, limit) 92 92 - if err != nil { 93 93 - return nil, err 94 94 - } 95 95 - defer rows.Close() 96 96 - 97 97 - var results []*PersonRecommendation 98 98 - for rows.Next() { 99 99 - rec := &PersonRecommendation{} 100 100 - if err := rows.Scan(&rec.DID, &rec.Handle, &rec.DisplayName, &rec.AvatarURL, 101 101 - &rec.Jaccard, &rec.CommonFeeds, &rec.CommonLikes, &rec.CommonTags); err != nil { 102 102 - return nil, err 103 103 - } 104 104 - results = append(results, rec) 105 105 - } 106 106 - return results, rows.Err() 107 107 - } 108 108 - 109 109 - func (e *Engine) GetArticleRecommendations(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) { 110 110 - rows, err := e.db.QueryContext(ctx, ` 111 111 - SELECT a.id, a.title, COALESCE(a.url, ''), r.feed_url, COALESCE(f.title, ''), 112 112 - COALESCE(a.author, ''), COALESCE(a.summary, ''), a.published, r.score 113 113 - FROM user_article_recommendations r 114 114 - JOIN articles a ON a.feed_url = r.feed_url AND a.url = r.article_url 115 115 - LEFT JOIN feeds f ON f.feed_url = r.feed_url 116 116 - WHERE r.user_did = ? 117 117 - ORDER BY r.score DESC 118 118 - LIMIT ? 119 119 - `, userDID, limit) 120 120 - if err != nil { 121 121 - return nil, err 122 122 - } 123 123 - defer rows.Close() 124 124 - 125 125 - var recs []*ArticleRecommendation 126 126 - for rows.Next() { 127 127 - rec := &ArticleRecommendation{} 128 128 - if err := rows.Scan(&rec.ArticleID, &rec.Title, &rec.URL, &rec.FeedURL, &rec.FeedTitle, 129 129 - &rec.Author, &rec.Summary, &rec.Published, &rec.Score); err != nil { 130 130 - return nil, err 131 131 - } 132 132 - recs = append(recs, rec) 133 133 - } 134 134 - return recs, rows.Err() 135 135 - } 136 136 - 137 137 - func (e *Engine) GetSimilarFeeds(ctx context.Context, feedURL string, limit int) ([]*SimilarFeed, error) { 138 138 - rows, err := e.db.QueryContext(ctx, ` 139 139 - SELECT f.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''), 140 140 - COALESCE(f.description, ''), COALESCE(f.feed_type, ''), sim.jaccard 141 141 - FROM ( 142 142 - SELECT feed_b AS peer_url, jaccard FROM feed_similarity WHERE feed_a = ? 143 143 - UNION ALL 144 144 - SELECT feed_a AS peer_url, jaccard FROM feed_similarity WHERE feed_b = ? 145 145 - ) sim 146 146 - JOIN feeds f ON f.feed_url = sim.peer_url 147 147 - ORDER BY sim.jaccard DESC 148 148 - LIMIT ? 149 149 - `, feedURL, feedURL, limit) 150 150 - if err != nil { 151 151 - return nil, err 152 152 - } 153 153 - defer rows.Close() 154 154 - 155 155 - var results []*SimilarFeed 156 156 - for rows.Next() { 157 157 - rec := &SimilarFeed{} 158 158 - if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, 159 159 - &rec.FeedType, &rec.Jaccard); err != nil { 160 160 - return nil, err 161 161 - } 162 162 - results = append(results, rec) 163 163 - } 164 164 - return results, rows.Err() 165 165 - }

+376

internal/cluster/scoring.go

reviewed

··· 1 1 + package cluster 2 2 + 3 3 + import ( 4 4 + "context" 5 5 + "database/sql" 6 6 + ) 7 7 + 8 8 + type FeedRecommendation struct { 9 9 + FeedURL string 10 10 + Title string 11 11 + SiteURL string 12 12 + Description string 13 13 + SubscriberCount int 14 14 + FaviconURL string 15 15 + Score float64 16 16 + } 17 17 + 18 18 + type PersonRecommendation struct { 19 19 + DID string 20 20 + Handle string 21 21 + DisplayName string 22 22 + AvatarURL string 23 23 + Jaccard float64 24 24 + CommonFeeds int 25 25 + CommonLikes int 26 26 + CommonTags int 27 27 + } 28 28 + 29 29 + type ArticleRecommendation struct { 30 30 + ArticleID int64 31 31 + Title string 32 32 + URL string 33 33 + FeedURL string 34 34 + FeedTitle string 35 35 + Author string 36 36 + Summary string 37 37 + Published sql.NullTime 38 38 + Score float64 39 39 + } 40 40 + 41 41 + func (e *Engine) GetFeedRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { 42 42 + subCount := 0 43 43 + _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount) 44 44 + 45 45 + if subCount < 5 { 46 46 + recs, err := e.ColdStartRecommendations(ctx, userDID, limit*2) 47 47 + if err == nil && len(recs) > 0 { 48 48 + return ApplyDiversity(recs, limit), nil 49 49 + } 50 50 + } 51 51 + 52 52 + recs, err := e.ComputeFeedRecommendationsOnDemand(ctx, userDID, limit*2) 53 53 + if err != nil { 54 54 + return nil, err 55 55 + } 56 56 + 57 57 + return ApplyDiversity(recs, limit), nil 58 58 + } 59 59 + 60 60 + func (e *Engine) GetPeopleRecommendations(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) { 61 61 + return e.ComputePeopleRecommendationsOnDemand(ctx, userDID, limit) 62 62 + } 63 63 + 64 64 + func (e *Engine) GetArticleRecommendations(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) { 65 65 + return e.ComputeArticleRecommendationsOnDemand(ctx, userDID, limit) 66 66 + } 67 67 + 68 68 + type SignalWeights struct { 69 69 + WSub float64 70 70 + WLike float64 71 71 + WTag float64 72 72 + WSocial float64 73 73 + WPop float64 74 74 + WCategory float64 75 75 + } 76 76 + 77 77 + func defaultWeights() SignalWeights { 78 78 + return SignalWeights{ 79 79 + WSub: 1.0, 80 80 + WLike: 0.5, 81 81 + WTag: 0.3, 82 82 + WSocial: 0.7, 83 83 + WPop: 0.2, 84 84 + WCategory: 0.4, 85 85 + } 86 86 + } 87 87 + 88 88 + func (e *Engine) GetWeights(ctx context.Context, userDID string) SignalWeights { 89 89 + w := defaultWeights() 90 90 + var dbW SignalWeights 91 91 + err := e.db.QueryRowContext(ctx, ` 92 92 + SELECT w_sub, w_like, w_tag, w_social, w_pop, w_category 93 93 + FROM user_signal_weights WHERE user_did = ? 94 94 + `, userDID).Scan(&dbW.WSub, &dbW.WLike, &dbW.WTag, &dbW.WSocial, &dbW.WPop, &dbW.WCategory) 95 95 + if err == nil { 96 96 + return dbW 97 97 + } 98 98 + return w 99 99 + } 100 100 + 101 101 + func (e *Engine) ComputeFeedRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { 102 102 + w := e.GetWeights(ctx, userDID) 103 103 + 104 104 + rows, err := e.db.QueryContext(ctx, ` 105 105 + WITH similar_users AS ( 106 106 + SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15 107 107 + UNION ALL 108 108 + SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15 109 109 + ), 110 110 + candidate_feeds AS ( 111 111 + SELECT s.feed_url, 112 112 + SUM(su.jaccard) AS sub_signal 113 113 + FROM similar_users su 114 114 + JOIN subscriptions s ON s.user_did = su.peer 115 115 + WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) 116 116 + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') 117 117 + GROUP BY s.feed_url 118 118 + ), 119 119 + like_signals AS ( 120 120 + SELECT s.feed_url, 121 121 + SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal 122 122 + FROM similar_users su 123 123 + JOIN likes l ON l.author_did = su.peer 124 124 + JOIN subscriptions s ON s.feed_url = l.feed_url 125 125 + WHERE s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) 126 126 + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') 127 127 + GROUP BY s.feed_url 128 128 + ), 129 129 + social_boost AS ( 130 130 + SELECT s.feed_url, 131 131 + SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social 132 132 + FROM follow_distances fd 133 133 + JOIN subscriptions s ON s.user_did = fd.user_b 134 134 + WHERE fd.user_a = ? 135 135 + AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) 136 136 + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') 137 137 + GROUP BY s.feed_url 138 138 + ), 139 139 + category_counts AS ( 140 140 + SELECT category, COUNT(*) AS cnt 141 141 + FROM subscriptions WHERE user_did = ? AND category IS NOT NULL AND category != '' 142 142 + GROUP BY category 143 143 + ), 144 144 + max_subs AS ( 145 145 + SELECT CAST(COALESCE(MAX(subscriber_count), 1) AS REAL) AS m FROM feeds 146 146 + ) 147 147 + SELECT cf.feed_url, COALESCE(f.title, ''), COALESCE(f.site_url, ''), 148 148 + COALESCE(f.description, ''), f.subscriber_count, COALESCE(f.favicon_url, ''), 149 149 + COALESCE(cf.sub_signal, 0) * ? 150 150 + + COALESCE(ls.like_signal, 0) * ? 151 151 + + COALESCE(sb.social, 0) * ? 152 152 + + COALESCE(LOG(1 + CAST(f.subscriber_count AS REAL)) / LOG(1 + ms.m), 0) * ? 153 153 + + CASE WHEN f.description IS NOT NULL AND EXISTS ( 154 154 + SELECT 1 FROM category_counts cc 155 155 + WHERE LOWER(f.description) LIKE '%' || LOWER(cc.category) || '%' 156 156 + ) THEN ? ELSE 0 END 157 157 + AS score 158 158 + FROM candidate_feeds cf 159 159 + JOIN feeds f ON f.feed_url = cf.feed_url 160 160 + LEFT JOIN like_signals ls ON ls.feed_url = cf.feed_url 161 161 + LEFT JOIN social_boost sb ON sb.feed_url = cf.feed_url 162 162 + CROSS JOIN max_subs ms 163 163 + ORDER BY score DESC 164 164 + LIMIT ? 165 165 + `, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, userDID, 166 166 + w.WSub, w.WLike, w.WSocial, w.WPop, w.WCategory, limit) 167 167 + if err != nil { 168 168 + return nil, err 169 169 + } 170 170 + defer rows.Close() 171 171 + 172 172 + var results []*FeedRecommendation 173 173 + for rows.Next() { 174 174 + rec := &FeedRecommendation{} 175 175 + if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, 176 176 + &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil { 177 177 + return nil, err 178 178 + } 179 179 + results = append(results, rec) 180 180 + } 181 181 + return results, rows.Err() 182 182 + } 183 183 + 184 184 + func (e *Engine) ComputeArticleRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*ArticleRecommendation, error) { 185 185 + w := e.GetWeights(ctx, userDID) 186 186 + 187 187 + rows, err := e.db.QueryContext(ctx, ` 188 188 + WITH similar_users AS ( 189 189 + SELECT user_b AS peer, jaccard FROM user_similarity WHERE user_a = ? AND jaccard > 0.15 190 190 + UNION ALL 191 191 + SELECT user_a AS peer, jaccard FROM user_similarity WHERE user_b = ? AND jaccard > 0.15 192 192 + ), 193 193 + liked_articles AS ( 194 194 + SELECT l.feed_url, l.article_url, 195 195 + SUM(su.jaccard * EXP(-0.023 * CAST(julianday('now') - julianday(l.created_at) AS REAL))) AS like_signal 196 196 + FROM similar_users su 197 197 + JOIN likes l ON l.author_did = su.peer 198 198 + WHERE NOT EXISTS ( 199 199 + SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url 200 200 + ) 201 201 + AND NOT EXISTS ( 202 202 + SELECT 1 FROM dismissed_recommendations d WHERE d.user_did = ? AND d.target_type = 'article' AND d.target_id = l.article_url 203 203 + ) 204 204 + GROUP BY l.feed_url, l.article_url 205 205 + ), 206 206 + social_likes AS ( 207 207 + SELECT l.feed_url, l.article_url, 208 208 + SUM(CASE WHEN fd.distance = 1 THEN 1.0 ELSE 0.3 END) AS social 209 209 + FROM follow_distances fd 210 210 + JOIN likes l ON l.author_did = fd.user_b 211 211 + WHERE fd.user_a = ? 212 212 + AND NOT EXISTS ( 213 213 + SELECT 1 FROM likes ul WHERE ul.author_did = ? AND ul.feed_url = l.feed_url AND ul.article_url = l.article_url 214 214 + ) 215 215 + GROUP BY l.feed_url, l.article_url 216 216 + ) 217 217 + SELECT a.id, a.title, COALESCE(a.url, ''), la.feed_url, COALESCE(f.title, ''), 218 218 + COALESCE(a.author, ''), COALESCE(a.summary, ''), a.published, 219 219 + COALESCE(la.like_signal, 0) * ? 220 220 + + COALESCE(sl.social, 0) * ? 221 221 + + EXP(-0.023 * CAST(julianday('now') - julianday(a.published) AS REAL)) * 0.2 222 222 + AS score 223 223 + FROM liked_articles la 224 224 + JOIN articles a ON a.feed_url = la.feed_url AND a.url = la.article_url 225 225 + LEFT JOIN feeds f ON f.feed_url = la.feed_url 226 226 + LEFT JOIN social_likes sl ON sl.feed_url = la.feed_url AND sl.article_url = la.article_url 227 227 + ORDER BY score DESC 228 228 + LIMIT ? 229 229 + `, userDID, userDID, userDID, userDID, userDID, userDID, w.WLike, w.WSocial, limit) 230 230 + if err != nil { 231 231 + return nil, err 232 232 + } 233 233 + defer rows.Close() 234 234 + 235 235 + var recs []*ArticleRecommendation 236 236 + for rows.Next() { 237 237 + rec := &ArticleRecommendation{} 238 238 + if err := rows.Scan(&rec.ArticleID, &rec.Title, &rec.URL, &rec.FeedURL, &rec.FeedTitle, 239 239 + &rec.Author, &rec.Summary, &rec.Published, &rec.Score); err != nil { 240 240 + return nil, err 241 241 + } 242 242 + recs = append(recs, rec) 243 243 + } 244 244 + return recs, rows.Err() 245 245 + } 246 246 + 247 247 + func (e *Engine) ComputePeopleRecommendationsOnDemand(ctx context.Context, userDID string, limit int) ([]*PersonRecommendation, error) { 248 248 + rows, err := e.db.QueryContext(ctx, ` 249 249 + SELECT u.did, u.handle, COALESCE(u.display_name, ''), COALESCE(u.avatar_url, ''), 250 250 + sim.jaccard, sim.common_feeds, COALESCE(sim.common_likes, 0), COALESCE(sim.common_tags, 0) 251 251 + FROM ( 252 252 + SELECT user_b AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_a = ? 253 253 + UNION ALL 254 254 + SELECT user_a AS peer_did, jaccard, common_feeds, common_likes, common_tags FROM user_similarity WHERE user_b = ? 255 255 + ) sim 256 256 + JOIN users u ON u.did = sim.peer_did 257 257 + WHERE u.handle IS NOT NULL AND u.handle != '' 258 258 + ORDER BY sim.jaccard DESC 259 259 + LIMIT ? 260 260 + `, userDID, userDID, limit) 261 261 + if err != nil { 262 262 + return nil, err 263 263 + } 264 264 + defer rows.Close() 265 265 + 266 266 + var results []*PersonRecommendation 267 267 + for rows.Next() { 268 268 + rec := &PersonRecommendation{} 269 269 + if err := rows.Scan(&rec.DID, &rec.Handle, &rec.DisplayName, &rec.AvatarURL, 270 270 + &rec.Jaccard, &rec.CommonFeeds, &rec.CommonLikes, &rec.CommonTags); err != nil { 271 271 + return nil, err 272 272 + } 273 273 + results = append(results, rec) 274 274 + } 275 275 + return results, rows.Err() 276 276 + } 277 277 + 278 278 + func (e *Engine) ComputeSignalProfiles(ctx context.Context) error { 279 279 + tx, err := e.db.BeginTx(ctx, nil) 280 280 + if err != nil { 281 281 + return err 282 282 + } 283 283 + defer func() { _ = tx.Rollback() }() 284 284 + 285 285 + if _, err := tx.ExecContext(ctx, `DELETE FROM user_signal_profiles`); err != nil { 286 286 + return err 287 287 + } 288 288 + 289 289 + _, err = tx.ExecContext(ctx, ` 290 290 + INSERT INTO user_signal_profiles (user_did, total_likes, total_tags, top_categories) 291 291 + SELECT 292 292 + u.did, 293 293 + (SELECT COUNT(*) FROM likes WHERE author_did = u.did), 294 294 + COALESCE((SELECT COUNT(DISTINCT TRIM(value)) 295 295 + FROM annotations, json_each('["' || REPLACE(tags, ',', '","') || '"]') 296 296 + WHERE author_did = u.did AND tags IS NOT NULL AND tags != '' 297 297 + ), 0), 298 298 + (SELECT '[' || GROUP_CONCAT('{"c":"' || category || '","n":"' || CAST(cnt AS TEXT) || '}') || ']' 299 299 + FROM ( 300 300 + SELECT category, COUNT(*) AS cnt 301 301 + FROM subscriptions WHERE user_did = u.did AND category IS NOT NULL AND category != '' 302 302 + GROUP BY category ORDER BY cnt DESC LIMIT 5 303 303 + ) 304 304 + ) 305 305 + FROM users u 306 306 + `) 307 307 + if err != nil { 308 308 + return err 309 309 + } 310 310 + 311 311 + e.logger.Info("signal profiles computed") 312 312 + return tx.Commit() 313 313 + } 314 314 + 315 315 + func (e *Engine) ColdStartRecommendations(ctx context.Context, userDID string, limit int) ([]*FeedRecommendation, error) { 316 316 + subCount := 0 317 317 + _ = e.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM subscriptions WHERE user_did = ?`, userDID).Scan(&subCount) 318 318 + if subCount >= 5 { 319 319 + return nil, nil 320 320 + } 321 321 + 322 322 + rows, err := e.db.QueryContext(ctx, ` 323 323 + WITH followed_feeds AS ( 324 324 + SELECT s.feed_url, 1.0 AS weight 325 325 + FROM follow_distances fd 326 326 + JOIN subscriptions s ON s.user_did = fd.user_b 327 327 + WHERE fd.user_a = ? AND fd.distance = 1 328 328 + AND s.feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) 329 329 + AND s.feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') 330 330 + ), 331 331 + popular_feeds AS ( 332 332 + SELECT feed_url, subscriber_count, 333 333 + LOG(1 + CAST(subscriber_count AS REAL)) / LOG(1 + CAST((SELECT COALESCE(MAX(subscriber_count), 1) FROM feeds) AS REAL)) AS pop_score 334 334 + FROM feeds 335 335 + WHERE subscriber_count > 0 336 336 + AND feed_url NOT IN (SELECT feed_url FROM subscriptions WHERE user_did = ?) 337 337 + AND feed_url NOT IN (SELECT target_id FROM dismissed_recommendations WHERE user_did = ? AND target_type = 'feed') 338 338 + ORDER BY subscriber_count DESC 339 339 + LIMIT 50 340 340 + ), 341 341 + all_candidates AS ( 342 342 + SELECT feed_url, MAX(weight) AS weight FROM ( 343 343 + SELECT feed_url, weight FROM followed_feeds 344 344 + UNION ALL 345 345 + SELECT feed_url, pop_score AS weight FROM popular_feeds 346 346 + ) 347 347 + GROUP BY feed_url 348 348 + ) 349 349 + SELECT ac.feed_url, 350 350 + COALESCE(f.title, ''), 351 351 + COALESCE(f.site_url, ''), 352 352 + COALESCE(f.description, ''), 353 353 + f.subscriber_count, 354 354 + COALESCE(f.favicon_url, ''), 355 355 + ac.weight AS score 356 356 + FROM all_candidates ac 357 357 + JOIN feeds f ON f.feed_url = ac.feed_url 358 358 + ORDER BY score DESC 359 359 + LIMIT ? 360 360 + `, userDID, userDID, userDID, userDID, userDID, limit) 361 361 + if err != nil { 362 362 + return nil, err 363 363 + } 364 364 + defer rows.Close() 365 365 + 366 366 + var results []*FeedRecommendation 367 367 + for rows.Next() { 368 368 + rec := &FeedRecommendation{} 369 369 + if err := rows.Scan(&rec.FeedURL, &rec.Title, &rec.SiteURL, &rec.Description, 370 370 + &rec.SubscriberCount, &rec.FaviconURL, &rec.Score); err != nil { 371 371 + return nil, err 372 372 + } 373 373 + results = append(results, rec) 374 374 + } 375 375 + return results, rows.Err() 376 376 + }

+63

internal/cluster/social.go

reviewed

··· 1 1 + package cluster 2 2 + 3 3 + import ( 4 4 + "context" 5 5 + ) 6 6 + 7 7 + func (e *Engine) ComputeFollowDistances(ctx context.Context) error { 8 8 + tx, err := e.db.BeginTx(ctx, nil) 9 9 + if err != nil { 10 10 + return err 11 11 + } 12 12 + defer func() { _ = tx.Rollback() }() 13 13 + 14 14 + if _, err := tx.ExecContext(ctx, `DELETE FROM follow_distances`); err != nil { 15 15 + return err 16 16 + } 17 17 + 18 18 + if _, err := tx.ExecContext(ctx, ` 19 19 + INSERT INTO follow_distances (user_a, user_b, distance) 20 20 + SELECT user_did, target_did, 1 21 21 + FROM follows 22 22 + WHERE user_did != target_did 23 23 + `); err != nil { 24 24 + return err 25 25 + } 26 26 + 27 27 + if _, err := tx.ExecContext(ctx, ` 28 28 + INSERT OR IGNORE INTO follow_distances (user_a, user_b, distance) 29 29 + SELECT f1.user_did, f2.target_did, 2 30 30 + FROM follows f1 31 31 + JOIN follows f2 ON f1.target_did = f2.user_did 32 32 + WHERE f1.user_did != f2.target_did 33 33 + `); err != nil { 34 34 + return err 35 35 + } 36 36 + 37 37 + e.logger.Info("follow distances computed") 38 38 + return tx.Commit() 39 39 + } 40 40 + 41 41 + func (e *Engine) ComputeFollowDistancesIncremental(ctx context.Context) error { 42 42 + var maxFollowed string 43 43 + err := e.db.QueryRowContext(ctx, ` 44 44 + SELECT COALESCE(MAX(followed_at), '1970-01-01') FROM follows 45 45 + `).Scan(&maxFollowed) 46 46 + if err != nil { 47 47 + return err 48 48 + } 49 49 + 50 50 + var lastComputed string 51 51 + err = e.db.QueryRowContext(ctx, ` 52 52 + SELECT COALESCE(MAX(updated_at), '1970-01-01') FROM user_similarity 53 53 + `).Scan(&lastComputed) 54 54 + if err != nil { 55 55 + return err 56 56 + } 57 57 + 58 58 + if maxFollowed <= lastComputed { 59 59 + return nil 60 60 + } 61 61 + 62 62 + return e.ComputeFollowDistances(ctx) 63 63 + }

+93

internal/cluster/weights.go

reviewed

··· 1 1 + package cluster 2 2 + 3 3 + import ( 4 4 + "context" 5 5 + ) 6 6 + 7 7 + const ( 8 8 + learningRate = 0.1 9 9 + weightMin = 0.1 10 10 + weightMax = 3.0 11 11 + minActionsTune = 5 12 12 + ) 13 13 + 14 14 + func (e *Engine) RewardSignal(ctx context.Context, userDID string, signal string) { 15 15 + e.adjustWeight(ctx, userDID, signal, 1.0) 16 16 + } 17 17 + 18 18 + func (e *Engine) PenalizeSignal(ctx context.Context, userDID string, signal string) { 19 19 + e.adjustWeight(ctx, userDID, signal, -1.0) 20 20 + } 21 21 + 22 22 + func (e *Engine) adjustWeight(ctx context.Context, userDID string, signal string, delta float64) { 23 23 + var actedCount int 24 24 + _ = e.db.QueryRowContext(ctx, ` 25 25 + SELECT COUNT(*) FROM recommendation_impressions WHERE user_did = ? AND acted = 1 26 26 + `, userDID).Scan(&actedCount) 27 27 + if actedCount < minActionsTune { 28 28 + return 29 29 + } 30 30 + 31 31 + var exists int 32 32 + _ = e.db.QueryRowContext(ctx, `SELECT 1 FROM user_signal_weights WHERE user_did = ?`, userDID).Scan(&exists) 33 33 + 34 34 + if exists == 0 { 35 35 + _, _ = e.db.ExecContext(ctx, ` 36 36 + INSERT INTO user_signal_weights (user_did, w_sub, w_like, w_tag, w_social, w_pop, w_category) 37 37 + VALUES (?, 1.0, 0.5, 0.3, 0.7, 0.2, 0.4) 38 38 + `, userDID) 39 39 + } 40 40 + 41 41 + column := signalToColumn(signal) 42 42 + if column == "" { 43 43 + return 44 44 + } 45 45 + 46 46 + adj := learningRate * delta 47 47 + _, _ = e.db.ExecContext(ctx, ` 48 48 + UPDATE user_signal_weights SET 49 49 + `+column+` = MAX(?, MIN(?, `+column+` * (1 + ?))), 50 50 + updated_at = CURRENT_TIMESTAMP 51 51 + WHERE user_did = ? 52 52 + `, weightMin, weightMax, adj, userDID) 53 53 + } 54 54 + 55 55 + func signalToColumn(signal string) string { 56 56 + switch signal { 57 57 + case "sub": 58 58 + return "w_sub" 59 59 + case "like": 60 60 + return "w_like" 61 61 + case "tag": 62 62 + return "w_tag" 63 63 + case "social": 64 64 + return "w_social" 65 65 + case "pop": 66 66 + return "w_pop" 67 67 + case "category": 68 68 + return "w_category" 69 69 + default: 70 70 + return "" 71 71 + } 72 72 + } 73 73 + 74 74 + func (e *Engine) GetDominantSignal(w SignalWeights) string { 75 75 + signals := map[string]float64{ 76 76 + "sub": w.WSub, 77 77 + "like": w.WLike, 78 78 + "tag": w.WTag, 79 79 + "social": w.WSocial, 80 80 + "pop": w.WPop, 81 81 + "category": w.WCategory, 82 82 + } 83 83 + 84 84 + var best string 85 85 + bestVal := -1.0 86 86 + for s, v := range signals { 87 87 + if v > bestVal { 88 88 + bestVal = v 89 89 + best = s 90 90 + } 91 91 + } 92 92 + return best 93 93 + }

+83 -16

internal/db/db.go

reviewed

··· 1 1 package db 2 2 3 3 import ( 4 4 + "context" 4 5 "database/sql" 5 5 - _ "github.com/mattn/go-sqlite3" 6 6 + "math" 6 7 "strings" 7 8 "time" 9 9 + 10 10 + "github.com/mattn/go-sqlite3" 8 11 ) 9 12 10 13 func NullStr(s string) sql.NullString { ··· 39 42 db.SetMaxOpenConns(1) 40 43 db.SetMaxIdleConns(2) 41 44 db.SetConnMaxLifetime(30 * time.Minute) 45 45 + 46 46 + conn, err := db.Conn(context.Background()) 47 47 + if err != nil { 48 48 + db.Close() 49 49 + return nil, err 50 50 + } 51 51 + 52 52 + err = conn.Raw(func(driverConn any) error { 53 53 + sqliteConn, ok := driverConn.(*sqlite3.SQLiteConn) 54 54 + if !ok { 55 55 + return nil 56 56 + } 57 57 + if err := sqliteConn.RegisterFunc("exp", func(x float64) float64 { return math.Exp(x) }, true); err != nil { 58 58 + return err 59 59 + } 60 60 + if err := sqliteConn.RegisterFunc("log", func(x float64) float64 { return math.Log(x) }, true); err != nil { 61 61 + return err 62 62 + } 63 63 + return nil 64 64 + }) 65 65 + _ = conn.Close() 66 66 + if err != nil { 67 67 + db.Close() 68 68 + return nil, err 69 69 + } 42 70 43 71 if err := initSchema(db); err != nil { 44 72 db.Close() ··· 158 186 PRIMARY KEY (user_a, user_b), 159 187 CHECK(user_a < user_b) 160 188 )`, 161 161 - `CREATE TABLE IF NOT EXISTS user_feed_recommendations ( 162 162 - user_did TEXT NOT NULL REFERENCES users(did), 163 163 - feed_url TEXT NOT NULL REFERENCES feeds(feed_url), 164 164 - score REAL NOT NULL, 165 165 - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, 166 166 - PRIMARY KEY (user_did, feed_url) 167 167 - )`, 168 168 - `CREATE TABLE IF NOT EXISTS user_article_recommendations ( 169 169 - user_did TEXT NOT NULL REFERENCES users(did), 170 170 - feed_url TEXT NOT NULL, 171 171 - article_url TEXT NOT NULL, 172 172 - score REAL NOT NULL, 173 173 - computed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, 174 174 - PRIMARY KEY (user_did, feed_url, article_url) 175 175 - )`, 176 189 `CREATE TABLE IF NOT EXISTS follows ( 177 190 user_did TEXT NOT NULL REFERENCES users(did), 178 191 target_did TEXT NOT NULL, ··· 208 221 `CREATE INDEX IF NOT EXISTS idx_follows_target ON follows(target_did)`, 209 222 `CREATE INDEX IF NOT EXISTS idx_follows_uri ON follows(uri)`, 210 223 `CREATE INDEX IF NOT EXISTS idx_user_similarity_b ON user_similarity(user_b)`, 224 224 + 225 225 + `CREATE TABLE IF NOT EXISTS dismissed_recommendations ( 226 226 + user_did TEXT NOT NULL REFERENCES users(did), 227 227 + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), 228 228 + target_id TEXT NOT NULL, 229 229 + reason TEXT, 230 230 + dismissed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, 231 231 + PRIMARY KEY (user_did, target_type, target_id) 232 232 + )`, 233 233 + 234 234 + `CREATE TABLE IF NOT EXISTS recommendation_impressions ( 235 235 + user_did TEXT NOT NULL REFERENCES users(did), 236 236 + target_type TEXT NOT NULL CHECK(target_type IN ('feed', 'article')), 237 237 + target_id TEXT NOT NULL, 238 238 + first_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, 239 239 + last_shown_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, 240 240 + shown_count INTEGER NOT NULL DEFAULT 1, 241 241 + acted BOOLEAN NOT NULL DEFAULT 0, 242 242 + PRIMARY KEY (user_did, target_type, target_id) 243 243 + )`, 244 244 + 245 245 + `CREATE TABLE IF NOT EXISTS follow_distances ( 246 246 + user_a TEXT NOT NULL, 247 247 + user_b TEXT NOT NULL, 248 248 + distance INTEGER NOT NULL CHECK(distance IN (1, 2)), 249 249 + PRIMARY KEY (user_a, user_b) 250 250 + )`, 251 251 + 252 252 + `CREATE TABLE IF NOT EXISTS user_signal_weights ( 253 253 + user_did TEXT PRIMARY KEY REFERENCES users(did), 254 254 + w_sub REAL NOT NULL DEFAULT 1.0, 255 255 + w_like REAL NOT NULL DEFAULT 0.5, 256 256 + w_tag REAL NOT NULL DEFAULT 0.3, 257 257 + w_social REAL NOT NULL DEFAULT 0.7, 258 258 + w_pop REAL NOT NULL DEFAULT 0.2, 259 259 + w_category REAL NOT NULL DEFAULT 0.4, 260 260 + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP 261 261 + )`, 262 262 + 263 263 + `CREATE TABLE IF NOT EXISTS user_signal_profiles ( 264 264 + user_did TEXT PRIMARY KEY REFERENCES users(did), 265 265 + total_likes INTEGER NOT NULL DEFAULT 0, 266 266 + total_tags INTEGER NOT NULL DEFAULT 0, 267 267 + top_categories TEXT, 268 268 + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP 269 269 + )`, 270 270 + 271 271 + `CREATE INDEX IF NOT EXISTS idx_dismissed_user_type ON dismissed_recommendations(user_did, target_type)`, 272 272 + `CREATE INDEX IF NOT EXISTS idx_impressions_user_unacted ON recommendation_impressions(user_did, acted, shown_count)`, 273 273 + `CREATE INDEX IF NOT EXISTS idx_impressions_last_shown ON recommendation_impressions(last_shown_at)`, 274 274 + `CREATE INDEX IF NOT EXISTS idx_follow_distances_b ON follow_distances(user_b)`, 275 275 + `CREATE INDEX IF NOT EXISTS idx_follow_distances_a_dist ON follow_distances(user_a, distance)`, 276 276 + `CREATE INDEX IF NOT EXISTS idx_likes_author_feed ON likes(author_did, feed_url, created_at)`, 277 277 + `CREATE INDEX IF NOT EXISTS idx_follows_followed_at ON follows(followed_at)`, 211 278 `CREATE INDEX IF NOT EXISTS idx_users_handle ON users(handle)`, 212 279 `CREATE VIRTUAL TABLE IF NOT EXISTS articles_fts USING fts5(title, summary, content, author, content=articles, content_rowid=id)`, 213 280 `CREATE TRIGGER IF NOT EXISTS articles_ai AFTER INSERT ON articles BEGIN

+6

internal/server/articles_handler.go

reviewed

··· 269 269 http.Error(w, err.Error(), http.StatusInternalServerError) 270 270 return 271 271 } 272 272 + _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String) 273 273 + sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID)) 274 274 + s.engine.RewardSignal(r.Context(), user.DID, sig) 272 275 } else { 273 276 like := &db.Like{ 274 277 URI: fmt.Sprintf("glean:like:%d", time.Now().UnixNano()), ··· 281 284 http.Error(w, err.Error(), http.StatusInternalServerError) 282 285 return 283 286 } 287 287 + _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "article", article.URL.String) 288 288 + sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID)) 289 289 + s.engine.RewardSignal(r.Context(), user.DID, sig) 284 290 } 285 291 } 286 292

+35

internal/server/dashboard_handler.go

reviewed

··· 3 3 import ( 4 4 "net/http" 5 5 "time" 6 6 + 7 7 + "pkg.rbrt.fr/glean/internal/cluster" 6 8 ) 7 9 8 10 func (s *Server) handleDashboard(w http.ResponseWriter, r *http.Request) { ··· 23 25 peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5) 24 26 feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 5) 25 27 28 28 + var impressions []cluster.Impression 29 29 + for _, rec := range feedRecs { 30 30 + impressions = append(impressions, cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL}) 31 31 + } 32 32 + for _, rec := range articleRecs { 33 33 + impressions = append(impressions, cluster.Impression{TargetType: "article", TargetID: rec.URL}) 34 34 + } 35 35 + if len(impressions) > 0 { 36 36 + _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions) 37 37 + } 38 38 + 26 39 since := time.Now().AddDate(0, 0, -7).Format(time.RFC3339) 27 40 personalTrending, _ := s.db.ListTrendingArticlesForUser(r.Context(), user.DID, since, 5, 0) 28 41 globalTrending, _ := s.db.ListTrendingArticles(r.Context(), user.DID, since, 10, 0) ··· 43 56 "Now": time.Now(), 44 57 }) 45 58 } 59 59 + 60 60 + func (s *Server) handleDismissArticleRecommendation(w http.ResponseWriter, r *http.Request) { 61 61 + user := currentUser(r) 62 62 + articleURL := r.FormValue("article_url") 63 63 + if articleURL == "" { 64 64 + http.Error(w, "article_url required", http.StatusBadRequest) 65 65 + return 66 66 + } 67 67 + 68 68 + reason := r.FormValue("reason") 69 69 + if reason == "" { 70 70 + reason = "not_interested" 71 71 + } 72 72 + 73 73 + if err := s.engine.DismissArticle(r.Context(), user.DID, articleURL, reason); err != nil { 74 74 + s.logger.Error("failed to dismiss article recommendation", "error", err) 75 75 + http.Error(w, err.Error(), http.StatusInternalServerError) 76 76 + return 77 77 + } 78 78 + 79 79 + w.WriteHeader(http.StatusOK) 80 80 + }

+35

internal/server/feeds_handler.go

reviewed

··· 9 9 "time" 10 10 11 11 "pkg.rbrt.fr/glean/internal/atproto" 12 12 + "pkg.rbrt.fr/glean/internal/cluster" 12 13 "pkg.rbrt.fr/glean/internal/db" 13 14 "pkg.rbrt.fr/glean/internal/feed" 14 15 ) ··· 28 29 allSubs, _ := s.db.ListSubscriptions(r.Context(), user.DID, "", 1000, 0) 29 30 feedRecs, _ := s.engine.GetFeedRecommendations(r.Context(), user.DID, 10) 30 31 peopleRecs, _ := s.engine.GetPeopleRecommendations(r.Context(), user.DID, 5) 32 32 + 33 33 + if len(feedRecs) > 0 { 34 34 + impressions := make([]cluster.Impression, len(feedRecs)) 35 35 + for i, rec := range feedRecs { 36 36 + impressions[i] = cluster.Impression{TargetType: "feed", TargetID: rec.FeedURL} 37 37 + } 38 38 + _ = s.engine.RecordImpressions(r.Context(), user.DID, impressions) 39 39 + } 31 40 deadFeeds, _ := s.db.ListDeadFeeds(r.Context(), user.DID, 7) 32 41 33 42 categories, _ := s.db.GetCategories(r.Context(), user.DID) ··· 121 130 http.Error(w, err.Error(), http.StatusInternalServerError) 122 131 return 123 132 } 133 133 + 134 134 + _ = s.engine.MarkImpressionActed(r.Context(), user.DID, "feed", feedURL) 135 135 + sig := s.engine.GetDominantSignal(s.engine.GetWeights(r.Context(), user.DID)) 136 136 + s.engine.RewardSignal(r.Context(), user.DID, sig) 124 137 125 138 sub, _ := s.db.GetSubscription(r.Context(), user.DID, feedURL) 126 139 if sub == nil { ··· 382 395 FeedURLs: result.FeedURLs, 383 396 Favicon: result.Favicon, 384 397 }) 398 398 + } 399 399 + 400 400 + func (s *Server) handleDismissFeedRecommendation(w http.ResponseWriter, r *http.Request) { 401 401 + user := currentUser(r) 402 402 + feedURL := r.FormValue("feed_url") 403 403 + if feedURL == "" { 404 404 + http.Error(w, "feed_url required", http.StatusBadRequest) 405 405 + return 406 406 + } 407 407 + 408 408 + reason := r.FormValue("reason") 409 409 + if reason == "" { 410 410 + reason = "not_interested" 411 411 + } 412 412 + 413 413 + if err := s.engine.DismissFeed(r.Context(), user.DID, feedURL, reason); err != nil { 414 414 + s.logger.Error("failed to dismiss feed recommendation", "error", err) 415 415 + http.Error(w, err.Error(), http.StatusInternalServerError) 416 416 + return 417 417 + } 418 418 + 419 419 + w.WriteHeader(http.StatusOK) 385 420 } 386 421 387 422 func nullString(s string) sql.NullString {

+3 -1

internal/server/server.go

reviewed

··· 162 162 r.Get("/list", s.handleFeedList) 163 163 r.Get("/discover-url", s.handleDiscoverFeedURL) 164 164 r.Post("/clear", s.handleClearAllSubscriptions) 165 165 + r.Post("/dismiss", s.handleDismissFeedRecommendation) 165 166 }) 166 167 167 168 s.router.Route("/articles", func(r chi.Router) { ··· 174 175 r.Post("/{id}/like", s.handleLikeArticle) 175 176 r.Post("/{id}/fetch-content", s.handleFetchContent) 176 177 r.Post("/mark-all-read", s.handleMarkAllRead) 178 178 + r.Post("/dismiss", s.handleDismissArticleRecommendation) 177 179 }) 178 180 179 181 s.router.Route("/trending", func(r chi.Router) { ··· 198 200 s.router.Post("/auth/logout", s.handleAuthLogout) 199 201 s.router.Get("/oauth/client-metadata", s.handleOAuthClientMetadata) 200 202 201 201 - xrpc := atproto.NewXRPCHandler(s.db.DB) 203 203 + xrpc := atproto.NewXRPCHandler(s.db.DB, s.engine) 202 204 s.router.Get("/xrpc/at.glean.listSubscriptions", xrpc.ListSubscriptions) 203 205 s.router.Get("/xrpc/at.glean.listAnnotations", xrpc.ListAnnotations) 204 206 s.router.Get("/xrpc/at.glean.listLikes", xrpc.ListLikes)

+9 -2

internal/tmpl/partials/recommendation-card.html

reviewed

··· 1 1 {{define "recommendation-card.html"}} 2 2 - <div class="bg-spot-surface rounded-xl p-3 hover:bg-spot-hover-50 transition"> 2 2 + <div class="recommendation-card bg-spot-surface rounded-xl p-3 hover:bg-spot-hover-50 transition"> 3 3 <div class="flex items-center justify-between gap-2"> 4 4 <a href="/articles?feed={{.feed_url}}" class="min-w-0 flex items-center gap-2 flex-1"> 5 5 {{if .favicon_url}}<img src="{{.favicon_url}}" class="w-4 h-4 rounded shrink-0" loading="lazy">{{end}} ··· 10 10 </a> 11 11 <div class="flex items-center gap-2 shrink-0"> 12 12 <span class="text-xs text-spot-secondary">{{.subscriber_count}} subs</span> 13 13 - <form hx-post="/feeds/add" hx-target="closest .bg-spot-surface" hx-swap="outerHTML" class="inline"> 13 13 + <form hx-post="/feeds/dismiss" hx-target="closest .recommendation-card" hx-swap="outerHTML" class="inline"> 14 14 + {{csrfInput .CSRFToken}} 15 15 + <input type="hidden" name="feed_url" value="{{.feed_url}}"> 16 16 + <button type="submit" title="Not interested" class="text-spot-muted hover:text-spot-red transition -m-1 p-1 rounded-full hover:bg-spot-hover-50 flex items-center"> 17 17 + <svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="w-4 h-4"><path stroke-linecap="round" stroke-linejoin="round" d="M6 18 18 6M6 6l12 12"/></svg> 18 18 + </button> 19 19 + </form> 20 20 + <form hx-post="/feeds/add" hx-target="closest .recommendation-card" hx-swap="outerHTML" class="inline"> 14 21 {{csrfInput .CSRFToken}} 15 22 <input type="hidden" name="feed_url" value="{{.feed_url}}"> 16 23 <button type="submit" class="text-xs font-bold uppercase tracking-button text-spot-green hover:brightness-110 transition">Subscribe</button>