Mirror of https://github.com/roostorg/coop github.com/roostorg/coop
0
fork

Configure Feed

Select the types of activity you want to include in your feed.

at main 418 lines 24 kB view raw view rendered
1# NCMEC Reporting 2 3Coop is integrated with the [CyberTipline Reporting API](https://report.cybertip.org/ispws/documentation) from the National Center for Missing and Exploited Children (NCMEC). Coop handles the full lifecycle: detecting known CSAM via hash matching or AI-based rules, routing content to a dedicated NCMEC manual review queue, and submitting CyberTips with the relevant metadata. 4 5> [!IMPORTANT] 6> NCMEC reporting requires your organization to be registered with NCMEC as an Electronic Service Provider (ESP). You can register at [esp.ncmec.org/registration](https://esp.ncmec.org/registration). 7 8## Prerequisites 9 10### CyberTip Reporting 11 12Before you can review and report content to NCMEC, you must have the following configured: 13 141. **CyberTipline API credentials**: a username and password for the CyberTipline API, obtained from NCMEC. 15 16#### Two separate sets of NCMEC credentials 17 18Coop's NCMEC integration uses two different NCMEC APIs, each requiring its own credentials: 19 20| | [Hash Sharing API](https://report.cybertip.org/ws-hashsharing/v2/documentation/) | [CyberTipline Reporting API](https://report.cybertip.org/ispws/documentation/index.html) | 21|---|---|---| 22| **Purpose** | Pull known CSAM hashes for matching | Submit CyberTip reports | 23| **URL** | `report.cybertip.org/ws-hashsharing` | `report.cybertip.org/ispws` | 24| **How to configure credentials** | In HMA's curator UI or via `TX_NCMEC_CREDENTIALS` env var on the HMA service | In Coop under **Settings → NCMEC** | 25 26Both sets of credentials must be obtained from NCMEC by [registering as an ESP](https://esp.ncmec.org/registration). They may be the same account or different accounts depending on how NCMEC provisions access for your organization. 27 282. **A User Item Type with a `creatorId` field**: NCMEC-type jobs are centered on a user, not individual pieces of content. Coop extracts the user from a content item via a `creatorId` field (a `RELATED_ITEM` field referencing the User Item Type). Coop then aggregates all media associated with that user into a single NCMEC review job. 29 303. **NCMEC org settings configured**: set via **Settings → NCMEC** (Admin only). See [NCMEC Settings](#ncmec-settings) below. 31 324. **A dedicated NCMEC manual review queue named "NCMEC Review**: Coop routes NCMEC jobs to the queue specified in your NCMEC settings. Queue IDs that are registered as production queues with Coop support will submit real CyberTips; all others use the NCMEC test environment. 33 345. **An Additional Info endpoint** (optional, but strongly recommended): a webhook Coop calls before submitting a CyberTip to fetch enriched metadata: email addresses, screen names, IP capture events, and per-media details. Without this, Coop submits the CyberTip with only the user ID and basic information from the Item data. 35 366. **A Preservation endpoint** (optional): a webhook Coop calls after a successful CyberTip submission so your platform can preserve relevant user data per NCMEC requirements. NOTE: Coop does not come with built-in preservation functionalities. 37 38### Hash Matching (HMA) 39 40If you want to automatically detect known CSAM via hash matching, you need a **separate** set of credentials for NCMEC's Hash Sharing API — distinct from the CyberTipline credentials above. See [Hash Banks in the User Guide](/docs/USER_GUIDE.md#hash-banks) for more detailed setup. Both credential sets are obtained from NCMEC when registering as an ESP, but they authenticate against different APIs and are configured in different places: Hash Sharing credentials go into HMA's own configuration (its curator UI or the `TX_NCMEC_CREDENTIALS` env var on the HMA service), while CyberTipline credentials are configured in the Coop settings UI. 41 42## NCMEC Settings 43 44Configure NCMEC reporting under **Settings � NCMEC** (`/dashboard/settings/ncmec`). 45 46![Setting up NCMEC reporting on Coop: add the required information for the reports submitted to NCMEC for content violating your company policies](./images/coop-ncmec-settings.png) 47 48| Setting | Description | 49|--------|-------------| 50| **Username** | Your NCMEC CyberTipline API username. | 51| **Password** | Your NCMEC CyberTipline API password. | 52| **Company Report Name** | Your organization name as it appears in NCMEC reports. This value is also sent as the ESP service name for the reported user in each CyberTip. | 53| **Legal URL** | URL to your Terms of Service or legal policies (e.g. `https://yourcompany.com/terms`). | 54| **Contact Email** | Email for the reporting person on the CyberTip. The XML receipt from NCMEC can serve as the ESP notification. | 55| **Terms of Service** | TOS text or URL to an acceptable use policy relevant to the incident being reported. Maximum 3000 characters. | 56| **Contact Person (for law enforcement)** | A contact person law enforcement can reach (other than the reporting contact email): first name, last name, email, phone. | 57| **More Info URL** | URL for additional information about your reporting process (e.g. `https://yourcompany.com/ncmec-info`). Used as the web page URL when "Default internet detail type" is set to "Web page." | 58| **Default NCMEC Queue** | When reviewers click "Enqueue to NCMEC," jobs are sent to this queue. Leave as "Use org default queue" to fall back to the organization's default queue. | 59| **Default Internet Detail Type** | The incident context (channel/medium) included in every CyberTip: Web page, Email, Newsgroup, Chat/IM, Online gaming, Cell phone, Non-internet, or Peer-to-peer. | 60| **NCMEC Additional Info Endpoint** | Webhook URL Coop calls before submitting a CyberTip to fetch enriched user and media metadata. See [Additional Info Endpoint](#additional-info-endpoint) below. Strongly recommended as without it, CyberTips are submitted with minimal user data. | 61| **NCMEC Preservation Endpoint** | Webhook URL Coop calls after a successful CyberTip submission with the report ID. See [Preservation Endpoint](#preservation-endpoint) below. | 62 63Saving credentials, Company Report Name, and Legal URL enables NCMEC reporting for the organization. The remaining fields are not required to submit a CyberTip, but filling them out makes reports significantly more actionable for NCMEC investigators. 64 65## Access and Roles 66 67NCMEC data is sensitive. Coop restricts access based on user roles: 68 69| Role | Can Access NCMEC Jobs | Can Submit CyberTips | Can Configure NCMEC Settings | 70|---|---|---|---| 71| Admin | Yes | Yes | Yes | 72| Moderator Manager | Yes | Yes | No | 73| Child Safety Moderator | Yes | Yes | No | 74| Moderator | No | No | No | 75| Analyst / Rules Manager | No | No | No | 76| External Moderator | No | No | No | 77 78## How Content Gets Routed to the NCMEC Queue 79 80There are four ways content can enter the NCMEC review queue: 81 82### 1. Hash Matching Known CSAM (HMA) 83 84Coop integrates with Meta's [Hasher-Matcher-Actioner (HMA)](https://github.com/facebook/ThreatExchange/tree/main/hasher-matcher-actioner), which matches uploaded media against NCMEC's database of known CSAM hashes. This is the most reliable detection path: a hash match is a strong signal that content is confirmed CSAM. 85 86#### How it works 87 88When media is submitted to your platform, Coop calls HMA to compute a perceptual hash (PDQ for images, MD5 for video) and checks it against the hash banks configured in HMA. If a match is found against an NCMEC-sourced bank, configure a routing rule to assign the content to your NCMEC review queue automatically. 89 90HMA syncs hashes from NCMEC via NCMEC's [Hash Sharing API](https://report.cybertip.org/ws-hashsharing/v2/documentation/), a separate API from the CyberTipline reporting API. The Hash Sharing API gives you access to NCMEC's database of image and video fingerprints for known CSAM, which HMA pulls on a schedule and indexes locally for fast matching. 91 92#### Setup 93 941. Configure NCMEC Hash Sharing API credentials in HMA (via HMA's curator UI or by setting the `TX_NCMEC_CREDENTIALS` environment variable on the HMA service). 952. In HMA, create a bank sourced from the NCMEC exchange. HMA will begin syncing hashes on its background fetch schedule (every 5 minutes by default). 963. In Coop, go to **Settings → Integrations** and add your HMA service URL. 974. In Coop's **Matching Banks**, the NCMEC-sourced bank will be available to reference in rules. 985. How you set things up depends on your use case: 99 100* If items are submitted by user reports (`POST /api/v1/report`): create a routing rule with the NCMEC hash match logic and assign it to the NCMEC queue. 101 102* If items are submitted via the items API `(POST /api/v1/items/async/)` and you want Coop to proactively flag matches without a user report: you need an automated enforcement rule with the image hash condition and a "Enqueue to NCMEC" action. 103 104 105### 2. Novel CSAM Detection (Content Safety API) 106 107For content that hasn't been seen before and therefore has no known hash, Coop integrates with Google's Content Safety API that classifies images for potential CSAM. You can configure a Routing Rule using a Content Safety signal to route high-confidence detections directly to the NCMEC queue, or to a triage queue for human review before escalation. 108 109### 3. Inbound Report Flagged as CSAM 110 111When your platform sends a user report to Coop's Report API with `reportedForReason.csam: true`, Coop automatically routes it to the NCMEC queue instead of the default review queue. These reasons should be configured by your reporting flow and match whatever reporting reasons you have defined. 112 113```json 114{ 115 "reporter": { "id": "user123", "typeId": "user-type-id" }, 116 "reportedAt": "2025-01-01T00:00:00Z", 117 "reportedItem": { 118 "id": "content456", 119 "typeId": "post-type-id", 120 "data": { ... } 121 }, 122 "reportedForReason": { 123 "csam": true 124 } 125} 126``` 127 128### 4. Manual Escalation from the Review Console 129 130In any review task, moderators with NCMEC access can select **Enqueue to NCMEC** from the action list. This immediately moves the job out of the current queue and into the NCMEC queue. 131 132### What Happens When Content Is Enqueued to NCMEC 133 134When any of the three triggers above fire, Coop: 135 1361. Identifies the **user** associated with the content (via the `creatorId` field on the content item, or directly if the item is a User type). 1372. Checks whether that user already has an open NCMEC job. If one exists, the existing job is updated with the new payload (the new job's content always takes precedence). No duplicate jobs are created. 1383. Fetches **all media** ever associated with that user across your platform. 1394. Creates a single consolidated NCMEC review job containing the user and all their media. 1405. Routes the job to the configured NCMEC queue (or the org default queue if none is configured). 141 142This user-centric aggregation means that even if a user has uploaded many pieces of CSAM, a single NCMEC review job is created for the reviewer, and a single CyberTip is submitted to NCMEC rather than separate reports per piece of content. 143 144 145## Reviewing an NCMEC Job 146 147The NCMEC job UI is distinct from standard review tasks. It is designed around the user and all of their associated media. 148 149![Coop's NCMEC Reporting task view showing aggregated media for a user, keyboard shortcuts for industry classifications, incident type dropdown, and per-media label selectors](./images/coop-ncmec-job.png) 150 151### Incident Type 152 153Select the applicable incident type from the NCMEC CyberTipline's defined categories: 154 155- Child Pornography (possession, manufacture, and distribution) 156- Child Sex Trafficking 157- Child Sex Tourism 158- Child Sexual Molestation 159- Misleading Domain Name 160- Misleading Words or Digital Images on the Internet 161- Online Enticement of Children for Sexual Acts 162- Unsolicited Obscene Material Sent to a Child 163 164### Industry Classification 165 166Apply an [ESP-designated industry classification](https://technologycoalition.org/wp-content/uploads/Tech_Coalition_Industry_Classification_System.pdf) to each media item being reported: 167 168| Classification | Description | 169|---|---| 170| **A1** | Prepubescent minor, explicit sexual activity | 171| **A2** | Prepubescent minor, non-explicit nudity or sexual posing | 172| **B1** | Pubescent minor, explicit sexual activity | 173| **B2** | Pubescent minor, non-explicit nudity or sexual posing | 174 175Keyboard shortcuts are available in the review UI to speed up classification. 176 177### File Annotations (Labels) 178 179Apply one or more labels to individual media items to provide NCMEC with additional context: 180 181| Label | Description | 182|---|---| 183| `animeDrawingVirtualHentai` | The file depicts anime, cartoon, virtual, or hentai content. | 184| `potentialMeme` | The file appears to be shared out of mimicry or other seemingly non-malicious intent. | 185| `viral` | The file is circulating rapidly from user to user. | 186| `possibleSelfProduction` | The file is believed to be self-produced. | 187| `physicalHarm` | The file depicts an intentional act of causing physical injury or trauma. | 188| `violenceGore` | The file depicts graphic violence or brutality. | 189| `bestiality` | The file involves an animal. | 190| `liveStreaming` | The content was streamed live at the time it was uploaded. | 191| `infant` | The file depicts an infant. | 192| `generativeAi` | The file is believed to be generated by AI. | 193 194### Submitting the CyberTip 195 196Once the reviewer has classified all media and selected the incident type, they click **Submit to NCMEC**. Coop then builds and submits the CyberTip automatically. See [CyberTip Submission Flow](#cybertip-submission-flow) below. 197 198 199## CyberTip Submission Flow 200 201When a reviewer submits a CyberTip, Coop performs the following steps: 202 2031. **Fetch additional info** — Coop calls your [Additional Info endpoint](#additional-info-endpoint) to retrieve enriched metadata: user email, screen name, IP capture events, and per-media details. 204 2052. **Build the CyberTip XML** — Coop assembles the full report: 206 207 - **escalateToHighPriority**: this is a boolean that marks the report as high priority (ie. there is abuse happening now) that NCMEC prioritizes when triaging. 208 - **Incident summary**: the incident type selected by the reviewer, and the timestamp of the most recently created media item as the `incidentDateTime`. 209 210 - **Internet details**: the channel or medium of the incident (e.g. Web page, Chat/IM, Email), set via the "Default Internet Detail Type" in NCMEC org settings. 211 212 - **Reporter**: your organization's name (`companyTemplate`), legal URL, contact email, optional terms of service language, and optional law enforcement contact person. All sourced from NCMEC org settings. 213 214 - **Reported user (`personOrUserReported`)**: the suspected perpetrator. Coop includes: 215 - `espIdentifier`: the user's internal platform ID 216 - `espService`: your organization's name (from `companyTemplate`) 217 - `screenName`: the user's username, from your Additional Info endpoint 218 - `displayName`: the user's display name, if available 219 - `email`: known email addresses for the user, from your Additional Info endpoint 220 - `ipCaptureEvent`: IP addresses associated with the user (e.g. login, upload events), from your Additional Info endpoint. Providing IP data significantly improves NCMEC's ability to identify and locate the suspect. 221 222 - **Victim**: if a child victim is identifiable (e.g. from a messaging context), Coop includes their `espIdentifier`, `screenName`, `displayName`, and `ipCaptureEvent`. This helps NCMEC locate and provide assistance to the victim. 223 2243. **Submit the report**: Coop POSTs the report XML to the NCMEC CyberTipline API and receives a `reportId`. 225 2264. **Upload media**: For each media item, Coop downloads the file from its URL and uploads it to NCMEC with full file metadata: 227 - Industry classification (A1/A2/B1/B2) 228 - File annotations (labels selected by the reviewer) 229 - IP capture events associated with the upload 230 - Whether the content was publicly available on your platform (`publiclyAvailable`) 231 - Whether the ESP viewed the file and its EXIF data (`fileViewedByEsp: true`, `exifViewedByEsp: true`) 232 - File hash, if provided by your Additional Info endpoint 233 2345. **Upload supplemental files**: Any additional files returned by your Additional Info endpoint (e.g. screenshots, supporting evidence) are uploaded to NCMEC as supplemental reported files. 235 2366. **Upload message threads**: If the user was reported in a messaging context, Coop generates a CSV for each conversation thread and uploads it to NCMEC. 237 2387. **Finalize the report**: Coop calls the NCMEC `/finish` endpoint to complete the submission. 239 2408. **Store the report**: The completed report (report ID, XML, all media details) is saved in Coop's database. 241 2429. **Send a preservation request**: If your org has a [Preservation endpoint](#preservation-endpoint) configured, Coop calls it with the report ID so you can preserve relevant user data. 243 244### Test vs. Production Submissions 245 246Coop determines whether to submit to the NCMEC test environment or production based on whether the NCMEC queue is registered as a production queue with Coop support. Test submissions go to `exttest.cybertip.org`; production submissions go to `report.cybertip.org`. 247 248 249## Webhooks 250 251### Additional Info Endpoint 252 253Coop calls this webhook **before** building a CyberTip to retrieve enriched metadata for the reported user and their media. This endpoint is optional but **strongly recommended** since without it, Coop submits the CyberTip with only the user's ID and whatever data was already sent to Coop via your Item API. 254 255Coop signs every request with your org's signing key. Verify the signature before processing. 256 257#### Request 258 259```json 260{ 261 "users": [ 262 { "id": "string", "typeId": "string" } 263 ], 264 "media": [ 265 { "id": "string", "typeId": "string" } 266 ] 267} 268``` 269 270#### Response 271 272```json 273{ 274 "users": [ 275 { 276 "id": "string", 277 "typeId": "string", 278 "screenName": "string", 279 "email": [ 280 { 281 "email": "user@example.com", 282 "type": "Home", 283 "verified": true, 284 "verificationDate": "2025-01-01T00:00:00Z" 285 } 286 ], 287 "ipCaptureEvent": [ 288 { 289 "ipAddress": "192.0.2.1", 290 "eventName": "Upload", 291 "dateTime": "2025-01-01T00:00:00Z", 292 "possibleProxy": false, 293 "port": 443 294 } 295 ], 296 "data": {} 297 } 298 ], 299 "media": [ 300 { 301 "id": "string", 302 "typeId": "string", 303 "missing": false, 304 "publiclyAvailable": true, 305 "fileName": "image.jpg", 306 "additionalInfo": ["string"], 307 "ipCaptureEvent": [ 308 { 309 "ipAddress": "192.0.2.1", 310 "eventName": "Upload", 311 "dateTime": "2025-01-01T00:00:00Z", 312 "possibleProxy": false, 313 "port": 443 314 } 315 ], 316 "fileDetails": { 317 "hash": "abee9985862d273160d930d2ac6ddb2cc33c74e73c702bcc8183d235f6f9685a", 318 "hashType": "PDQ" 319 } 320 } 321 ], 322 "additionalFiles": [ 323 { 324 "fileUrl": "https://yourplatform.com/evidence/file.pdf", 325 "fileName": "evidence.pdf", 326 "additionalInfo": ["Supporting evidence"] 327 } 328 ], 329 "messages": [ 330 { "id": "string", "typeId": "string", "ipAddress": "192.0.2.1" } 331 ], 332 "additionalInfo": "string" 333} 334``` 335 336#### Response Fields 337 338| Field | Type | Description | 339|---|---|---| 340| `users` | Array | Must include an entry for every user in the request. | 341| `users.id` | String | Must match the `id` from the request. | 342| `users.typeId` | String | Must match the `typeId` from the request. | 343| `users.screenName` | String | The user's screen name or username on your platform. | 344| `users.email` | Array | Known email addresses for the user. `type` may be `Business`, `Home`, or `Work`. | 345| `users.ipCaptureEvent` | Array | IP events associated with the user (e.g. logins, registrations). `eventName` may be `Login`, `Registration`, `Purchase`, `Upload`, `Other`, or `Unknown`. | 346| `users.data` | Object | Raw item data for the user. | 347| `media` | Array | Must include an entry for every media item in the request if present. | 348| `media.id` | String | Must match the `id` from the request. | 349| `media.typeId` | String | Must match the `typeId` from the request. | 350| `media.missing` | Boolean | Set to `true` if the media is no longer available. If **all** media items are `missing: true`, no CyberTip is filed. | 351| `media.publiclyAvailable` | Boolean | Whether the media was publicly accessible on your platform at the time of reporting. | 352| `media.fileName` | String | Original filename of the media. | 353| `media.additionalInfo` | Array\<String\> | Additional context about the media to include in the NCMEC file details. | 354| `media.ipCaptureEvent` | Array | IP events associated with this media item (e.g. the upload event). | 355| `media.fileDetails` | Object | Hash information for the file: `{ hash, hashType }`. | 356| `additionalFiles` | Array | Extra files to upload to NCMEC as supplemental evidence (e.g. screenshots). | 357| `messages` | Array | Message-level IP address data for conversation thread context. | 358| `additionalInfo` | String | Top-level freeform additional information to include in the report. | 359 360> [!IMPORTANT] 361> If the response does not include an entry for every user and media item in the request, Coop will throw an error and not submit the CyberTip. Your endpoint must return a response entry for each requested user and media item. 362> If all media items have `missing: true`, Coop will not file the CyberTip. The job will be marked as a permanent error and will not be retried. 363 364 365### Preservation Endpoint 366 367Platforms that submit CyberTips may have data preservation obligations under laws like [18 U.S.C. § 368 2258A](https://uscode.house.gov/view.xhtml?req=granuleid:USC-prelim-title18-section2258A) and the [REPORT Act 369 (2024)](https://www.missingkids.org/blog/2024/first-line-of-defense-guidelines-to-help-online-platforms-detect-sexually-exploited-kids), which extended content retention 370 requirements to one year. Talk to your legal team to understand your organization's specific obligations. 371 372 Coop calls a preservation endpoint you build and host immediately after a CyberTip is successfully submitted. Your endpoint should trigger whatever internal workflow 373 handles data retention — for example, flagging the account for legal hold, snapshotting relevant records, or notifying your legal team. Coop passes the reported user, the 374 media included in the CyberTip, and the NCMEC-assigned report ID so you have everything you need to identify what to retain. 375 Coop signs every request with your org's signing key. Verify the signature before processing. 376 377#### Request 378 379```json 380{ 381 "user": { "id": "string", "typeId": "string" }, 382 "reportedMedia": [ 383 { "id": "string", "typeId": "string" } 384 ], 385 "reportId": "string" 386} 387``` 388 389| Field | Description | 390|---|---| 391| `user` | The user who was reported to NCMEC. | 392| `reportedMedia` | All media items that were included in the CyberTip. | 393| `reportId` | The NCMEC-assigned CyberTip report ID. | 394 395Coop only checks for a successful HTTP status code. The response body is ignored. This webhook is only called for production (non-test) CyberTip submissions. 396 397 398## Retry Behavior 399 400If a CyberTip submission fails (e.g. due to a transient network error or NCMEC API outage), Coop automatically retries the submission. A background job runs periodically and retries any failed NCMEC decisions that: 401 402* Have not already been successfully submitted (no matching report in the database) 403* Have fewer than 10 prior retry attempts 404* Are not marked as a permanent error (e.g. all media missing) 405* Were decided within the past 30 days 406 407 408## Viewing Submitted Reports 409 410After a CyberTip is submitted, the report is stored in Coop and accessible from the NCMEC Reports dashboard. The report record includes: 411 412* The NCMEC-assigned report ID 413* The reported user 414* All media included in the report 415* The full CyberTip XML 416* Any supplemental files uploaded 417* Any conversation thread CSVs uploaded 418* Whether the submission was a test or production report