Twitter Circles was a feature (May 2022 - October 2023) that let users share tweets with a limited audience. When you download your Twitter archive, Circle tweets are included but not marked as such - there's no field indicating a tweet was Circle-only.

This is a privacy issue: if you share or upload your archive, your Circle tweets become public.

The Solution#

This tool uses Twitter's public syndication API to detect Circle tweets:

Public tweets return __typename: "Tweet" with full data
Circle tweets return __typename: "TweetTombstone" with "This Post is unavailable"

The syndication API (cdn.syndication.twimg.com/tweet-result) is the same endpoint used for embedded tweets on websites - no API key or authentication required.

Usage#

Step 1: Detect Circle Tweets#

cd circle-filter
bun install
bun run src/detect.ts --archive ../data

This will:

Load all tweets from the Circle era (May 2022 - Nov 2023)
Skip retweets (can't be Circle tweets)
Check each tweet against the syndication API (20 concurrent requests)
Save progress after each batch (resumable if interrupted)

Runtime: ~10 minutes for a large archive (32k tweets in the Circle era with 20 concurrent requests)

Output:

output/circle-tweets.json - Array of Circle tweet IDs
output/detection-log.json - Full log of every API check
output/progress.json - Resume checkpoint

Step 2: Generate Cleaned Archive#

bun run src/clean.ts --archive ../data --circles ./output/circle-tweets.json

This generates cleaned tweets*.js files in output/ with Circle tweets removed.

Output:

output/tweets.js - Cleaned tweets (same format as original)
output/tweets-part1.js, output/tweets-part2.js, etc.
output/clean-summary.json - Summary of removal

For Future Archives#

Once you have circle-tweets.json, reuse it for new archive exports:

bun run src/clean.ts --archive /path/to/new/archive/data --circles ./output/circle-tweets.json

No need to re-run detection - your Circle tweet IDs won't change.

How Detection Works#

For each tweet in Circle era (May 2022 - Nov 2023):
  1. Skip if retweet (can't be Circle)
  2. Skip if in deleted-tweets.js (deleted, not Circle)
  3. Query: cdn.syndication.twimg.com/tweet-result?id={id}&token={token}
  4. If response.__typename === "TweetTombstone" → Circle tweet
  5. Save to circle-tweets.json

The token is calculated as (id / 1e15) * π in base-36. Tweet IDs exceed Number.MAX_SAFE_INTEGER, so we use BigInt with hi/lo split to preserve precision:

const bigId = BigInt(id);
const hi = Number(bigId / 1_000_000_000_000_000n);
const lo = Number(bigId % 1_000_000_000_000_000n) / 1e15;
token = ((hi + lo) * Math.PI).toString(36).replace(/(0+|\.)/g, '');

Rate Limiting#

Concurrent requests: 20
On HTTP 429: Exponential backoff (1s, 2s, 4s, 8s...)
Max retries per tweet: 5
Request timeout: 30s (prevents hung connections)
Progress saved after each batch for resume after interruption
Transient errors (429, 5xx, network) are retried on resume

Verification#

The detection-log.json file contains every API call made:

{
    "startedAt": "2025-12-07T...",
    "completedAt": "2025-12-07T...",
    "totalCandidates": 32150,
    "circleCount": 847,
    "results": {
        "1718307594148651356": {
            "id": "1718307594148651356",
            "typename": "TweetTombstone",
            "tombstoneText": "This Post is unavailable. Learn more",
            "retries": 0,
            "timestamp": "2025-12-07T..."
        }
    }
}

Development#

bun run typecheck    # TypeScript 7 (tsgo)
bun run lint         # ESLint 9 with strict rules
bun run format       # Prettier
bun run knip         # Dead code detection

File Structure#

circle-filter/
├── src/
│   ├── types.ts         # TypeScript interfaces
│   ├── syndication.ts   # API client with retry logic
│   ├── utils.ts         # Archive loading helpers
│   ├── detect.ts        # Main detection script
│   └── clean.ts         # Archive cleaner script
├── output/              # Generated files go here
├── eslint.config.mjs
├── knip.json
├── package.json
├── tsconfig.json
└── README.md

Credits#

Detection method based on research into the Twitter syndication API. See also:

License#

MIT

Configure Feed