Mirror of https://github.com/roostorg/osprey
github.com/roostorg/osprey
modernize some docs (#38)
+17
-11
+8
-1
README.md
+8
-1
README.md
···
11
11
12
12
This repository is home to one of ROOST’s safety tools, the Osprey rules engine. Osprey is an open-source event stream decisions engine and analysis UI designed to investigate and take automatic action on events and their properties as they happen in real-time. Originally developed internally at [Discord](https://discord.com/) to combat spam, abuse, botting, and scripting across its platform, Osprey has now been open-sourced to help other platforms facing similar challenges.
13
13
14
-
Osprey is a library for processing actions through human written rules and outputting labels, webhooks back to an API and other sinks. It evaluates events using structured logic (SML), user-defined functions (UDFs), and external signals to assign labels, verdicts, and actions.
14
+
Osprey is a library for processing actions through human-written rules and outputting verdicts & custom effects back to configurable output sinks. It evaluates events using structured rule logic (SML) that is extendable via user-defined functions (UDFs). Osprey can also track state across events by labelling entities if implementers provide a labels service backend (see [labels_service.py](./example_plugins/src/labels_service.py) for a Postgres-backed labels service example)
15
15
16
16
This 'Rules \+ Investigation' tool is able to:
17
17
···
83
83
```bash
84
84
# Test linting
85
85
uv run ruff check
86
+
uv run mypy .
86
87
87
88
# Test formatting
88
89
uv run ruff format --diff
···
91
92
uv run pre-commit run --all-files
92
93
93
94
5. **Start Services:**
95
+
94
96
```bash
95
97
docker compose up -d
96
98
```
99
+
100
+
6. (Optional) **Port Forward the UI/UI API:**
101
+
102
+
If you are running the docker compose on a headless machine, you will need to port forward the UI and UI API.
103
+
Namely, ports `5002` (UI) and `5004` (UI API). Then, you can connect via http://localhost:5002/ :D
97
104
98
105
99
106
### Development Workflow
+9
-10
docs/DEVELOPMENT.md
+9
-10
docs/DEVELOPMENT.md
···
113
113
docker compose up -d
114
114
```
115
115
116
-
This starts up four services:
117
-
- **Kafka** (KRaft mode): Message streaming for user generated events
116
+
This starts up many services, including:
118
117
- **Osprey Worker**: The main engine that processes input events given the rules and UDFs
119
-
- **Test Data Producer**: Optional with `--profile test_data`
118
+
- **Test Data Producer**: Optional with `--profile test_data`
119
+
- **Osprey UI**: Frontend service that hosts the react code for the web interface and communicates to the UI API
120
120
- **Osprey UI API**: Backend service that provides data and functionality to the web interface
121
+
- **Kafka** (KRaft mode): Message streaming for user generated events
122
+
- **Postgres**: A database that the Worker, UI API, and Druid use for various reasons, such as the Postgres-backed Labels Service (in the example plugins)
123
+
- **Druid**: A database that consumes Osprey Worker outputs to power the UI API for real-time querying
121
124
122
-
### 6. Start the UI (in a new terminal):
123
-
``` bash
124
-
cd osprey_ui
125
-
npm install
126
-
npm start
127
-
```
128
125
129
-
### 7. Access the Application
126
+
### 6. Access the Application
130
127
The UI will automatically connect to the backend services running in Docker containers.
131
128
132
129
- Osprey UI: http://localhost:5002
···
202
199
203
200
# Type checking (on specific files/modules)
204
201
uv run mypy osprey_worker/src/osprey_worker/lib
202
+
# Or you can type check every module (this will happen in CI)
203
+
uv run mypy .
205
204
206
205
# Run all pre-commit hooks
207
206
uv run pre-commit run --all-files