experiments in a post-browser web
1# Feeds & Series API
2
3This document covers the APIs for time-series data and external feeds in Peek.
4
5## Overview
6
7Peek uses two item types for temporal data:
8
9- **`series`** - Scalar values over time (numeric or text)
10- **`feed`** - Entries from external sources (RSS, APIs, logs)
11
12Both store their data in the `item_events` table. The item defines *what* you're tracking; events record *when* things happened.
13
14## Core Concepts
15
16### Items = Definitions
17
18Items with `type: 'series'` or `type: 'feed'` define what you're tracking:
19
20```javascript
21// A series tracks scalar values
22const series = await api.datastore.addItem('series', {
23 content: 'AAPL Stock Price',
24 metadata: JSON.stringify({ unit: 'USD', source: 'yahoo-finance' })
25});
26
27// A feed tracks entries from a source
28const feed = await api.datastore.addItem('feed', {
29 content: 'https://example.com/rss',
30 metadata: JSON.stringify({ name: 'Example Blog', format: 'rss' })
31});
32```
33
34### Events = Facts
35
36Events are append-only records of what happened:
37
38```javascript
39// Record a series observation
40await api.datastore.addItemEvent(series.data.id, {
41 value: 156.32, // numeric value
42 occurredAt: Date.now()
43});
44
45// Record a feed entry
46await api.datastore.addItemEvent(feed.data.id, {
47 content: 'https://example.com/article-123', // entry URL
48 occurredAt: pubDate,
49 metadata: JSON.stringify({ title: 'Article Title', author: 'Jane' })
50});
51```
52
53## API Reference
54
55### Item Operations
56
57Standard item CRUD works for series and feeds:
58
59```javascript
60// Create
61const result = await api.datastore.addItem('series', { content: 'Mood', metadata: '{}' });
62
63// Read
64const item = await api.datastore.getItem(itemId);
65
66// Update
67await api.datastore.updateItem(itemId, { metadata: '{"unit":"points"}' });
68
69// Delete (also delete events first)
70await api.datastore.deleteItemEvents(itemId);
71await api.datastore.deleteItem(itemId);
72
73// Query all series
74const series = await api.datastore.queryItems({ type: 'series' });
75
76// Query all feeds
77const feeds = await api.datastore.queryItems({ type: 'feed' });
78```
79
80### Event Operations
81
82#### `addItemEvent(itemId, options)`
83
84Add an event to a series or feed.
85
86```javascript
87await api.datastore.addItemEvent(itemId, {
88 content: 'text value or URL', // optional
89 value: 42, // optional, for numeric data
90 occurredAt: Date.now(), // optional, defaults to now
91 metadata: JSON.stringify({...}) // optional, JSON string
92});
93```
94
95#### `getItemEvent(eventId)`
96
97Get a single event by ID.
98
99```javascript
100const event = await api.datastore.getItemEvent(eventId);
101// { id, itemId, content, value, occurredAt, metadata, createdAt }
102```
103
104#### `queryItemEvents(filter)`
105
106Query events with filters.
107
108```javascript
109const events = await api.datastore.queryItemEvents({
110 itemId: 'item_123', // single item
111 // OR
112 itemIds: ['item_1', 'item_2'], // multiple items
113 since: Date.now() - 86400000, // after timestamp
114 until: Date.now(), // before timestamp
115 limit: 50, // max results
116 offset: 0, // skip first N
117 order: 'desc' // 'asc' or 'desc' by occurredAt
118});
119```
120
121#### `deleteItemEvent(eventId)`
122
123Delete a single event.
124
125```javascript
126await api.datastore.deleteItemEvent(eventId);
127```
128
129#### `deleteItemEvents(itemId)`
130
131Delete all events for an item.
132
133```javascript
134const count = await api.datastore.deleteItemEvents(itemId);
135```
136
137#### `getLatestItemEvent(itemId)`
138
139Get the most recent event.
140
141```javascript
142const latest = await api.datastore.getLatestItemEvent(itemId);
143```
144
145#### `countItemEvents(itemId, filter)`
146
147Count events for an item.
148
149```javascript
150const count = await api.datastore.countItemEvents(itemId, {
151 since: Date.now() - 86400000 // last 24 hours
152});
153```
154
155## Use-Case Examples
156
157### 1. Stock Price Tracking
158
159Track numeric values over time from a web source.
160
161```javascript
162// Create the series
163const stock = await api.datastore.addItem('series', {
164 content: 'AAPL',
165 metadata: JSON.stringify({
166 name: 'Apple Stock Price',
167 unit: 'USD',
168 source_url: 'https://finance.yahoo.com/quote/AAPL'
169 })
170});
171const stockId = stock.data.id;
172
173// Record observations (from polling job)
174async function recordPrice(price) {
175 await api.datastore.addItemEvent(stockId, {
176 value: price,
177 occurredAt: Date.now()
178 });
179}
180
181// Query last 7 days
182const prices = await api.datastore.queryItemEvents({
183 itemId: stockId,
184 since: Date.now() - 7 * 24 * 60 * 60 * 1000,
185 order: 'asc'
186});
187
188// Calculate average
189const avg = prices.data.reduce((sum, e) => sum + e.value, 0) / prices.data.length;
190
191// Get latest price
192const latest = await api.datastore.getLatestItemEvent(stockId);
193console.log(`Current: $${latest.data.value}, 7-day avg: $${avg.toFixed(2)}`);
194```
195
196### 2. Mood Tracking (Text Values)
197
198Track non-numeric values like mood or status.
199
200```javascript
201// Create the series
202const mood = await api.datastore.addItem('series', {
203 content: 'Mood',
204 metadata: JSON.stringify({ input: 'tag-board' })
205});
206const moodId = mood.data.id;
207
208// Record mood (from tag button board)
209async function recordMood(moodValue) {
210 await api.datastore.addItemEvent(moodId, {
211 content: moodValue, // 'great', 'good', 'meh', 'bad'
212 occurredAt: Date.now()
213 });
214}
215
216// Usage
217await recordMood('great');
218await recordMood('meh');
219
220// Query this week's moods
221const moods = await api.datastore.queryItemEvents({
222 itemId: moodId,
223 since: Date.now() - 7 * 24 * 60 * 60 * 1000
224});
225
226// Count by mood
227const counts = {};
228for (const event of moods.data) {
229 counts[event.content] = (counts[event.content] || 0) + 1;
230}
231console.log('Mood distribution:', counts);
232```
233
234### 3. Habit Tracking with Streaks
235
236Track daily habits and calculate streaks.
237
238```javascript
239// Create the series
240const pushups = await api.datastore.addItem('series', {
241 content: 'Pushups',
242 metadata: JSON.stringify({ frequency: 'daily', tag: 'pushups' })
243});
244const pushupId = pushups.data.id;
245
246// Record daily check-in
247async function logPushups(count) {
248 const today = new Date().toISOString().split('T')[0]; // YYYY-MM-DD
249 await api.datastore.addItemEvent(pushupId, {
250 value: count,
251 occurredAt: Date.now(),
252 metadata: JSON.stringify({ date: today })
253 });
254}
255
256// Calculate current streak
257async function getStreak() {
258 const events = await api.datastore.queryItemEvents({
259 itemId: pushupId,
260 limit: 365,
261 order: 'desc'
262 });
263
264 if (!events.success || events.data.length === 0) return 0;
265
266 let streak = 0;
267 let expectedDate = new Date();
268 expectedDate.setHours(0, 0, 0, 0);
269
270 for (const event of events.data) {
271 const meta = JSON.parse(event.metadata || '{}');
272 const eventDate = new Date(meta.date);
273 eventDate.setHours(0, 0, 0, 0);
274
275 // Check if this is the expected date (consecutive day)
276 const diffDays = Math.round((expectedDate - eventDate) / 86400000);
277
278 if (diffDays === 0) {
279 streak++;
280 expectedDate.setDate(expectedDate.getDate() - 1);
281 } else if (diffDays === 1) {
282 // Skip today if not logged yet
283 expectedDate.setDate(expectedDate.getDate() - 1);
284 if (expectedDate.getTime() === eventDate.getTime()) {
285 streak++;
286 expectedDate.setDate(expectedDate.getDate() - 1);
287 } else {
288 break;
289 }
290 } else {
291 break;
292 }
293 }
294
295 return streak;
296}
297
298// Usage
299await logPushups(20);
300const streak = await getStreak();
301console.log(`Current streak: ${streak} days`);
302
303// Get personal best
304const allEvents = await api.datastore.queryItemEvents({ itemId: pushupId });
305const max = Math.max(...allEvents.data.map(e => e.value || 0));
306console.log(`Personal best: ${max} pushups`);
307```
308
309### 4. Ticket Availability Trigger
310
311Poll a page and notify when condition is met.
312
313```javascript
314// Create the series (acts as trigger definition)
315const tickets = await api.datastore.addItem('series', {
316 content: 'Taylor Swift Tickets',
317 metadata: JSON.stringify({
318 source_url: 'https://ticketmaster.com/event/123',
319 selector: '.buy-button',
320 condition: 'exists',
321 notified: false
322 })
323});
324const ticketId = tickets.data.id;
325
326// Polling function (run periodically)
327async function checkTickets() {
328 const item = await api.datastore.getItem(ticketId);
329 const meta = JSON.parse(item.data.metadata || '{}');
330
331 if (meta.notified) return; // Already notified
332
333 // In a real extension, you'd load the page in a hidden window
334 // and run document.querySelector(meta.selector)
335 const available = await checkPageForElement(meta.source_url, meta.selector);
336
337 // Record the check
338 await api.datastore.addItemEvent(ticketId, {
339 value: available ? 1 : 0,
340 content: available ? 'Available!' : 'Not available',
341 occurredAt: Date.now()
342 });
343
344 // Trigger notification
345 if (available) {
346 meta.notified = true;
347 await api.datastore.updateItem(ticketId, {
348 metadata: JSON.stringify(meta)
349 });
350
351 // Show notification
352 new Notification('Tickets Available!', {
353 body: 'Taylor Swift tickets are now on sale!'
354 });
355 }
356}
357```
358
359### 5. RSS Feed Reader
360
361Subscribe to and poll RSS feeds.
362
363```javascript
364// Subscribe to a feed
365async function subscribe(url) {
366 const response = await fetch(url);
367 const xml = await response.text();
368 const title = parseFeedTitle(xml); // Your XML parsing function
369
370 const feed = await api.datastore.addItem('feed', {
371 content: url,
372 metadata: JSON.stringify({
373 name: title,
374 format: 'rss',
375 lastPolledAt: 0
376 })
377 });
378
379 return feed.data.id;
380}
381
382// Poll a feed for new entries
383async function pollFeed(feedId) {
384 const item = await api.datastore.getItem(feedId);
385 const feedUrl = item.data.content;
386
387 const response = await fetch(feedUrl);
388 const xml = await response.text();
389 const entries = parseFeedEntries(xml); // Your XML parsing function
390
391 // Get existing entries to dedupe
392 const existing = await api.datastore.queryItemEvents({ itemId: feedId, limit: 100 });
393 const existingGuids = new Set(
394 existing.data.map(e => JSON.parse(e.metadata || '{}').guid)
395 );
396
397 // Add new entries
398 for (const entry of entries) {
399 if (existingGuids.has(entry.guid)) continue;
400
401 await api.datastore.addItemEvent(feedId, {
402 content: entry.link,
403 occurredAt: entry.pubDate,
404 metadata: JSON.stringify({
405 guid: entry.guid,
406 title: entry.title,
407 author: entry.author
408 })
409 });
410 }
411
412 // Update last polled
413 const meta = JSON.parse(item.data.metadata || '{}');
414 meta.lastPolledAt = Date.now();
415 await api.datastore.updateItem(feedId, { metadata: JSON.stringify(meta) });
416}
417
418// Get all entries across feeds
419async function getAllEntries(limit = 50) {
420 const feeds = await api.datastore.queryItems({ type: 'feed' });
421 const feedIds = feeds.data.map(f => f.id);
422
423 const entries = await api.datastore.queryItemEvents({
424 itemIds: feedIds,
425 limit: limit,
426 order: 'desc'
427 });
428
429 return entries.data.map(e => ({
430 ...e,
431 meta: JSON.parse(e.metadata || '{}')
432 }));
433}
434```
435
436### 6. System Metrics / Action Log
437
438Track internal application events.
439
440```javascript
441// Create a log feed
442const actionLog = await api.datastore.addItem('feed', {
443 content: 'peek:system:actions',
444 metadata: JSON.stringify({ format: 'internal' })
445});
446const logId = actionLog.data.id;
447
448// Log an action
449async function logAction(action, details = {}) {
450 await api.datastore.addItemEvent(logId, {
451 content: action,
452 occurredAt: Date.now(),
453 metadata: JSON.stringify(details)
454 });
455}
456
457// Usage
458await logAction('page_open', { url: 'https://example.com', source: 'bookmark' });
459await logAction('settings_changed', { key: 'theme', value: 'dark' });
460await logAction('extension_loaded', { id: 'feeds', version: '1.0.0' });
461
462// Query recent actions
463const recent = await api.datastore.queryItemEvents({
464 itemId: logId,
465 since: Date.now() - 3600000, // last hour
466 order: 'desc'
467});
468
469// Count actions by type
470const counts = {};
471for (const event of recent.data) {
472 counts[event.content] = (counts[event.content] || 0) + 1;
473}
474console.log('Action counts:', counts);
475```
476
477### 7. Spotify Listening History
478
479Pull data from external APIs.
480
481```javascript
482// Create the feed
483const spotify = await api.datastore.addItem('feed', {
484 content: 'spotify:me:recently-played',
485 metadata: JSON.stringify({
486 format: 'api',
487 service: 'spotify',
488 lastSyncedAt: 0
489 })
490});
491const spotifyId = spotify.data.id;
492
493// Sync from Spotify API (requires OAuth token)
494async function syncSpotify(accessToken) {
495 const response = await fetch('https://api.spotify.com/v1/me/player/recently-played?limit=50', {
496 headers: { 'Authorization': `Bearer ${accessToken}` }
497 });
498 const data = await response.json();
499
500 // Get existing to dedupe
501 const existing = await api.datastore.queryItemEvents({ itemId: spotifyId, limit: 100 });
502 const existingIds = new Set(
503 existing.data.map(e => JSON.parse(e.metadata || '{}').playedAt)
504 );
505
506 // Add new plays
507 for (const item of data.items) {
508 const playedAt = new Date(item.played_at).getTime();
509 if (existingIds.has(item.played_at)) continue;
510
511 await api.datastore.addItemEvent(spotifyId, {
512 content: item.track.uri,
513 occurredAt: playedAt,
514 metadata: JSON.stringify({
515 playedAt: item.played_at,
516 trackName: item.track.name,
517 artistName: item.track.artists[0]?.name,
518 albumName: item.track.album?.name,
519 durationMs: item.track.duration_ms
520 })
521 });
522 }
523
524 // Update sync time
525 const meta = JSON.parse((await api.datastore.getItem(spotifyId)).data.metadata || '{}');
526 meta.lastSyncedAt = Date.now();
527 await api.datastore.updateItem(spotifyId, { metadata: JSON.stringify(meta) });
528}
529
530// Get top artists from listening history
531async function getTopArtists(days = 30) {
532 const plays = await api.datastore.queryItemEvents({
533 itemId: spotifyId,
534 since: Date.now() - days * 24 * 60 * 60 * 1000
535 });
536
537 const artistCounts = {};
538 for (const play of plays.data) {
539 const meta = JSON.parse(play.metadata || '{}');
540 const artist = meta.artistName || 'Unknown';
541 artistCounts[artist] = (artistCounts[artist] || 0) + 1;
542 }
543
544 return Object.entries(artistCounts)
545 .sort((a, b) => b[1] - a[1])
546 .slice(0, 10);
547}
548```
549
550## Schema Reference
551
552### items table (existing, extended)
553
554```sql
555items (
556 id TEXT PRIMARY KEY,
557 type TEXT NOT NULL, -- 'url', 'text', 'tagset', 'image', 'series', 'feed'
558 content TEXT, -- identifier, URL, or name
559 metadata TEXT, -- JSON with type-specific data
560 ...
561)
562```
563
564### item_events table (new)
565
566```sql
567item_events (
568 id TEXT PRIMARY KEY,
569 itemId TEXT NOT NULL, -- FK to series or feed
570 content TEXT, -- text value, URL, or message
571 value REAL, -- numeric value (optional)
572 occurredAt INTEGER, -- when it happened (Unix ms)
573 metadata TEXT, -- JSON extras
574 createdAt INTEGER -- when recorded
575)
576
577-- Indexes
578CREATE INDEX idx_item_events_item_time ON item_events(itemId, occurredAt DESC);
579CREATE INDEX idx_item_events_occurred ON item_events(occurredAt DESC);
580```
581
582## Best Practices
583
5841. **Use `value` for numbers, `content` for text/URLs** - Makes queries and aggregations simpler.
585
5862. **Store dedup keys in metadata** - For feeds, store `guid` to avoid duplicate entries.
587
5883. **Set `occurredAt` to event time, not record time** - For RSS entries, use `pubDate`; for API data, use the original timestamp.
589
5904. **Keep metadata lightweight** - Truncate descriptions, store only what you need for display.
591
5925. **Delete events when deleting items** - Always call `deleteItemEvents(itemId)` before `deleteItem(itemId)`.
593
5946. **Use `order: 'desc'` for recent-first** - Most UIs want newest first.
595
5967. **Batch queries with `itemIds`** - When showing entries from multiple feeds, query once with all IDs.