Mirror of https://github.com/roostorg/osprey
github.com/roostorg/osprey
1# Osprey User Interface Guide
2
3## Getting Started
4
5```bash
6cd osprey_ui
7npm install
8npm start
9```
10
11The Osprey UI has several pages accessible by a left-hand menu:
12
13
14
15Home will bring you to the default page of Osprey, with three main columns.
16
17
18
19### Left Column: Query
20
21#### Query Box
22
23The Osprey Query UI uses the same SML syntax as rules, but for searching and filtering near-real-time and historical data rather than creating new rules. Using the test data generator, you can try writing a query to look for an action called “create\_post” specifically from a given User ID.
24
25
26
27You can also use a UDF in your query. If you ever forget what a UDF does, you can hover on the information symbol for a tip:
28
29
30
31A query can be run against a time window ranging from the last second to the last 3 months (and also a custom range):
32
33
34
35The Osprey UI is designed to be dynamic and update in real-time. If any other component in the other two columns is interacted with, the query will automatically update and vice versa. The query also automatically populates the URL. This can be handy for sharing a specific query with someone on a team, but may present privacy risks.
36
37
38
39#### History
40
41Every query is logged in the Query History view, and there is a dropdown filter to only show queries that you have run.
42
43When you hover over the query, it will also show the Top N Charts used during the query session (more on that below).
44
45
46
47The Query History can also be accessed and seen in a different format via the left-side menu. From here you can filter by the user who ran the query, view the original query, and run it using the same time range the original query used.
48
49
50
51#### Saved Queries
52
53If there are specific queries that are used often, Osprey provides the ability to save a query:
54
55
56
57The user who initiated the query and when the query was first run is logged as part of the Saved Query. Saved Queries can also be accessed via the left-side menu. The user who saved the query and what time it was saved is logged and visible. There is a drop-down menu at the top to filter saved queries by users.
58
59
60
61### Middle Column: Charts
62
63The middle column in Osprey shows two types of charts: **Time Series** and **Top N Results**. Both sections provide the ability to add extra charts to see different slices of time or types of top results.
64
65
66
67#### Time Series Chart
68
69The Time Series chart shows a visualization of the results in the query over a period of time. The time ranges include:
70
71* Minute
72* Fifteen minutes
73* Half hour
74* Hour
75* Day
76* Week
77* Month
78
79Hovering over a bar in the time series chart shows how many events took place during that time.
80
81
82
83There is also a time and date picker above the time series chart where you can set a custom range:
84
85
86
87An extra table can be added for another view of a different unit of time. To get rid of the table, you can “[yeet](https://www.urbandictionary.com/define.php?term=Yeet) it”.
88
89
90
91#### Top N Results
92
93Adding a Top N Results table populates a table with the top results for the results of the query. You can view and assign labels to a specific entity by hovering over it and clicking “Edit Labels”
94
95
96
97
98
99You can also select PoP (Period over Period) to compare the query results with results from a window of time in the past to see the delta.
100
101
102
103### Right Column: Event Stream
104
105The Event Stream is essentially Osprey's "live feed" and investigation dashboard where security teams can:
106
107* Monitor real-time activity
108* Search historical events using SML queries
109* Investigate suspicious patterns
110* Track rule execution results
111* Drill down into specific users/entities
112
113It provides a more detailed view of each event that matches the query. The Event Stream can show metadata related to accounts that can link to other internal tools that provide detailed information about an account and/or further enforcement actions.
114
115
116The event stream is also viewable in a card format vs a list format (list format shown in the screenshot).
117
118Osprey users may have personal preferences on how to do investigations and what information is most helpful for them. Osprey makes it easy to customize the types of information shown in the Event Stream by clicking “Summary Features”
119
120
121### Labeling
122
123Any unique entity can be labeled in the Osprey UI. This manual labeling tool is used by Safety teams to tag individual entities (users, IPs, emails, etc.) with labels. Labels are essentially the manual annotation tool that feeds into Osprey's automated rule system, allowing human judgment to enhance machine detection. Labels can be positive, negative, or neutral. Examples:
124
125**Negative Labels: Harmful/problematic behavior**
126* Examples: "spammer", "bot", "banned", "suspicious"
127
128**Positive Labels: Good/trusted behavior**
129* Examples: "verified", "trusted", "premium\_user"
130
131**Neutral Labels: Informational tags**
132* Examples: "new\_user", "from\_mobile", "beta\_tester"
133
134
135
136
137### UDF Documentation
138
139The UDF Documentation page can be accessed via the left-side menu. It dynamically updates based on the code, so any new UDFs added will show up on this page. This page essentially serves as the "API reference" for the SML language, making it easy for users to discover and properly use all available functions when writing rules and queries.
140
141
142This page can be used as a manual for writing SML rules or queries, guide for understanding parameter types and requirements, and act as a plugin discovery portal to explore what custom UDFs are loaded.
143
144### Bulk Labeling
145
146There are two ways to bulk label items in Osprey: the left-side menu and via the chart column. In this example, you can bulk label all the users that have posted a message that is not empty:
147
148
149
150**Bulk labels can be dangerous if there’s a false positive\!** Osprey provides a counter of how many unique entities are about to be bulk labeled at the top. Labels can be positive, negative, or neutral. A reason must be provided when labeling anything. Each bulk job will create a unique task ID and log the user who initiated the bulk job, the status of the bulk labeling, and a link to the query that the bulk job originated from.
151
152To view all bulk labeling jobs that have been done, click into “Bulk Job History” from the left-side menu. You’ll need the unique task ID to look up a bulk job.
153
154
155
156### Rule Visualizer
157
158The Rule Visualizer shows how upstream labels, rules, and downstream labels interact with one another. To use it, select an Action or a Label. A graph view will appear showing the relations between rules and labels.
159
160* **Red circle:** label upstream of a rule
161* **Blue square:** rule
162* **Green circle:** label downstream of a rule
163
164
165
166### Query Syntax
167
168#### Actions
169
170Actions are events that are sent to Osprey. An event is simply something that happens. When a user does something like create a post, send a message, change their username, etc an event happens to represent that. There are probably a lot of events emitted in your org, and Osprey doesn’t need to consume all of them.
171
172##### Features
173
174A feature is any variable in the global namespace in Osprey. All features must be uniquely named. However, prefixing a \`\_\` at the start of a variable name prevents it from being exported as a feature and keeps the variable within the local file’s namespace.
175
176Features are outputs of Osprey executions. Downstream, they are sent to and indexed by Druid, so users can query for events based on feature names later, i.e. \`UserEmail \== '`` despicable@example.com` ``.
177
178```py
179UserId: Entity[int] = EntityJson(type='User', path='$.user.id', coerce_type=True)
180
181UserEmail: str = JsonData(type='Email', path='$.user.email', required=False)
182```
183
184In the example above, both `UserId` and `UserEmail` are features.
185
186##### Entities
187
188Entities are a special type of Feature. All entities are features, but not all features are entities. An entity can have effects applied to it, such as labels, classifications, or signals). Every entity has a type that determines which effects can be applied to it based on static validations.
189
190Entities get special treatment within the Osprey UI. Clicking on an entity in the tool will take you to an Entity View, providing a deep dive into its history.
191
192This could be:
193
194* User ID
195* IP Address
196* Post Text
197* Internet Service Provider
198
199#### Effects
200
201Effects can be triggered when one or many rules are evaluated to be true. These are validated and handled in aggregate at the end of an execution output. For example, an effect might apply a label to an entity marking it as a “Spammer”.
202
203### Basic Query Structure
204
205Let’s say you have a data field called “EventType” and one of the events is to “create\_post”. You are looking for posts by a user whose ID is 12345 and you don’t want to see empty posts. Your query would look like:
206
207```py
208# Simple field comparisons
209EventType == "create_post"
210UserId == 12345
211MessageText != Null
212```
213
214#### Combining Conditions
215
216Let’s say you’re looking for any matches where a user tried to login more than 3 times. You can create a query to check for two types of data fields: “EventType” and “LoginAttempts”.
217
218If you’re looking for multiple types of events in “EventType” like for posts AND messages, you can use brackets to list the types of events.
219
220```py
221 # Multiple conditions (AND)
222 EventType == "user_login" and LoginAttempts >= 3
223
224 # OR conditions
225 EventType in ["create_post", "send_message"]
226 (UserId == 123) or (UserId == 456)
227```
228
229#### Using UDFs in Queries
230
231UDFs (read more [here](rules.md#user-defined-functions-udfs)) are a powerful part of queries. Once you define a UDF with the specific desired logic, you can reference it in a query.
232
233> [!NOTE]
234> If you try to query a UDF that doesn’t exist, Osprey will silently fail with a 500 error.
235
236```py
237 # Text search
238 TextContains(text=PostContent, phrase="spam")
239 RegexMatch(target=MessageText, pattern="(buy|sell|deal)")
240
241 # List operations
242 ListLength(list=UserConnections) > 10
243```
244
245#### Label Queries
246
247Since the UI searches across actions/events:
248
249* **Don't use:** HasLabel() - won't work in Query UI
250* **Use instead**: DidAddLabel() - shows when an action added a label
251
252```py
253 # Find actions that added specific labels
254 DidAddLabel(entity_type="UserId", label_name="likely_spammer")
255 DidAddLabel(entity_type="IpAddress", label_name="suspicious")
256```
257
258### Example Queries
259
260```py
261# Find suspicious login attempts:
262 EventType == "user_login" and LoginAttempts >= 5
263
264# Find posts containing specific words:
265 EventType == "create_post" and TextContains(text=PostContent, phrase="urgent")
266
267# Find users who were flagged:
268 DidAddLabel(entity_type="UserId", label_name="flagged")
269
270# Complex search:
271 EventType == "send_message" and
272 RegexMatch(target=MessageText, pattern="(click|link|urgent)") and
273 not DidAddLabel(entity_type="UserId", label_name="verified")