CLI utility to ingest embedded json metadata from yt-dlp downloads to a SQLite database file
yt-dlp
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

feat: Add a basic `video_metadata_view` view for SQLite for easier parsing

0xBA5E64 05b3da47 653997e5

+13 -1
+2
migrations/20260418204924_add-video-metadata-view.down.sql
··· 1 + -- Add down migration script here 2 + DROP VIEW IF EXISTS video_metadata_view;
+10
migrations/20260418204924_add-video-metadata-view.up.sql
··· 1 + -- Add up migration script here 2 + CREATE VIEW video_metadata_view AS SELECT 3 + metadata->>'fulltitle' title, 4 + metadata->>'duration' duration, 5 + metadata->>'channel' channel, 6 + metadata->>'id' AS id, 7 + metadata->>'upload_date' AS upload_date, 8 + metadata->>'view_count' AS views, 9 + video_path 10 + FROM videos ORDER BY concat(video_path, title) DESC;
+1 -1
readme.md
··· 26 26 ### Why not deconstruct the json metadata into database columns? 27 27 The youtube-dl / yt-dlp json metadata schema does not appear to be stable, and as such, **all data could not reliably be serialized into standardized columns**. Different versions of these tools may also have generated and attached different data. Since SQLite makes it trivial to work with json, and in an effort to preserve as much of this data as possible, the whole of the json data is parsed, validated, and inserted into the database. 28 28 29 - However, for convenience, it may make sense to offer a [table view](https://sqlite.org/lang_createview.html) that extracts much of the metadata as fields as so ease in querying 29 + However, for convenience, the `video_metadata_view` [table view](https://sqlite.org/lang_createview.html) may also be used for ease of querying. 30 30 31 31 ### Why SQLite? Why not a Key-value store database? or a document database? 32 32 With the current schema consisting of only two colums: `video_path` & `metadata`, one might argue this job would be a better fit for a Key-value database, like Redis or one of it's many open-source derivatives, or a document database such as MongoDB to better fit the unstructured nature of the json metadata.