A monorepo containing jupyter-blocks and jupyter-tidyblocks. Blockly extension for JupyterLab.
Update documentation
+883
-192
+16
CHANGELOG.md
+16
CHANGELOG.md
···
74
74
- Migrated from Yarn 4 to npm; `yarn.lock` / `.yarnrc.yml` / `.yarn/` removed; `"resolutions"` → `"overrides"`; `jlpm` replaced with `npm run` in all scripts
75
75
- `yarn.lock` added to `.gitignore` (regenerated as a build side-effect by `@jupyterlab/builder`'s bundled jlpm)
76
76
77
+
### Built-in datasets
78
+
79
+
Three new built-in datasets added to the **Data** category:
80
+
81
+
- **iris** — Fisher iris dataset (sepal/petal measurements for 3 species) via `sns.load_dataset('iris')`
82
+
- **titanic** — Titanic passenger survival dataset via `sns.load_dataset('titanic')`
83
+
- **gapminder** — Life expectancy / GDP across countries and years via `px.data.gapminder()`
84
+
85
+
No new dependencies required — iris and titanic use the existing seaborn import;
86
+
gapminder uses the existing plotly.express import.
87
+
77
88
### Docs
78
89
79
90
- `docs/getting-started.md`: step-by-step guide for installing and testing the extension in JupyterLab
80
91
- `docs/architecture.md`: full architecture reference (package layout, data-flow, extension points)
81
92
- `docs/blocks-reference.md`: complete block reference with dplyr mapping, description, and generated Python for every block
93
+
- `docs/custom-blocks.md`: step-by-step tutorial for adding custom blocks and toolboxes
82
94
- `docs/work-summary.md`: narrative summary of all engineering work done in this release
83
95
- `docs/modernization-plan.md`: full modernization plan with phase-by-phase status
96
+
- `docs/installation.md`: updated to use npm commands and correct package name
97
+
- `docs/other_extensions.md`: updated to reference `jupyter-tidyblocks` package and point to `custom-blocks.md`
98
+
- `docs/toolbox.md`: updated with correct category list and colours
99
+
- `docs/index.rst`: TOC updated to include all new docs
84
100
- `README.md`: rewritten; credits Greg Wilson's tidyblocks and QuantStack/jupyterlab-blockly
85
101
86
102
<!-- <END NEW CHANGELOG ENTRY> -->
+305
docs/custom-blocks.md
+305
docs/custom-blocks.md
···
1
+
# Adding Custom Blocks
2
+
3
+
This guide walks you through adding a new block to jupyter-tidyblocks from scratch.
4
+
By the end you will have a working block that appears in the toolbox, can be dragged
5
+
onto the canvas, and generates executable Python code.
6
+
7
+
We will build a concrete example: a **`normalize` block** that scales a numeric
8
+
column to the 0–1 range using pandas `min-max` normalization.
9
+
10
+
---
11
+
12
+
## Overview
13
+
14
+
Every block requires three things:
15
+
16
+
1. **Block shape** — the visual appearance and inputs, defined in
17
+
`packages/tidyblocks/src/blocks/`.
18
+
2. **Python generator** — a function that converts the block to Python code, defined
19
+
in `packages/tidyblocks/src/generators/python/`.
20
+
3. **Toolbox entry** — so the block appears in the sidebar panel, defined in
21
+
`packages/tidyblocks/src/toolbox.ts`.
22
+
23
+
Optionally, if the block needs a Python import (e.g. `import scipy`), you also set
24
+
a **`toplevel_init`** string on the block.
25
+
26
+
---
27
+
28
+
## Step 1 — Define the block shape
29
+
30
+
Block shapes live in `packages/tidyblocks/src/blocks/`. Each file in this directory
31
+
corresponds to a category. Our block is a transform operation, so open
32
+
`transform.ts`.
33
+
34
+
Add a new entry inside the `Blockly.defineBlocksWithJsonArray([...])` call:
35
+
36
+
```typescript
37
+
{
38
+
// Type name must be unique across the whole Blockly registry.
39
+
// Convention: tidyblocks_<category>_<verb>
40
+
type: 'tidyblocks_transform_normalize',
41
+
42
+
// message0 is the label text. %1 marks where a field or input goes.
43
+
message0: 'normalize column %1',
44
+
45
+
// args0 defines the inputs referenced in message0.
46
+
// field_input is a plain text box. Other options: field_number,
47
+
// field_dropdown, input_value (for connectable value blocks).
48
+
args0: [
49
+
{ type: 'field_input', name: 'COLUMN', text: 'value' }
50
+
],
51
+
52
+
// previousStatement/nextStatement make the block chainable.
53
+
// Set both to null to allow connecting above and below.
54
+
// Remove one to make the block a source (top) or terminal (bottom).
55
+
previousStatement: null,
56
+
nextStatement: null,
57
+
58
+
// Colour should match the category. Transform blocks use #76AADB.
59
+
colour: '#76AADB',
60
+
61
+
tooltip: 'Scale a numeric column to the 0–1 range (min-max normalization).'
62
+
},
63
+
```
64
+
65
+
### Field types quick-reference
66
+
67
+
| Field type | When to use | Key properties |
68
+
|---|---|---|
69
+
| `field_input` | Free-text string (column name, label) | `name`, `text` (default) |
70
+
| `field_number` | Numeric value with optional min/max | `name`, `value`, `min`, `max`, `precision` |
71
+
| `field_dropdown` | Enumerated choice | `name`, `options: [['Label', 'VALUE'], ...]` |
72
+
| `input_value` | Connectable value block (expression) | `name`, `check` (e.g. `'Boolean'`) |
73
+
74
+
---
75
+
76
+
## Step 2 — Write the Python generator
77
+
78
+
Generators live in `packages/tidyblocks/src/generators/python/`. Open the file
79
+
matching your category (`transform.ts`).
80
+
81
+
Add a new `forBlock` entry:
82
+
83
+
```typescript
84
+
// dplyr: no direct equivalent — min-max scaling
85
+
pythonGenerator.forBlock['tidyblocks_transform_normalize'] = block => {
86
+
const col = block.getFieldValue('COLUMN');
87
+
return (
88
+
`_df = _df.assign(**{'${col}': ` +
89
+
`(_df['${col}'] - _df['${col}'].min()) / ` +
90
+
`(_df['${col}'].max() - _df['${col}'].min())})\n`
91
+
);
92
+
};
93
+
```
94
+
95
+
The generator receives the `block` object (and optionally a `generator` reference
96
+
for nested value blocks). It returns a **string of Python code** ending with `\n`.
97
+
98
+
### Reading block values
99
+
100
+
| Scenario | Code |
101
+
|---|---|
102
+
| Text / number field | `block.getFieldValue('FIELD_NAME')` |
103
+
| Connected value block | `generator.valueToCode(block, 'INPUT_NAME', Order.NONE)` |
104
+
| Dropdown field | `block.getFieldValue('FIELD_NAME')` — returns the option's `VALUE` string |
105
+
106
+
### The `_df` convention
107
+
108
+
All transform blocks read from `_df` and write back to `_df`. This is what makes
109
+
blocks chainable — each block in a stack operates on the DataFrame left by the block
110
+
above it. Source (data) blocks create `_df`; terminal blocks (display, plot, save)
111
+
consume it without writing it back.
112
+
113
+
---
114
+
115
+
## Step 3 — Add a `toplevel_init` (if needed)
116
+
117
+
If your block requires a Python import that isn't already guaranteed by the data
118
+
block's preamble (`pandas`, `numpy`, `seaborn`), attach a `toplevel_init` string
119
+
to the block after `defineBlocksWithJsonArray`:
120
+
121
+
```typescript
122
+
// Only needed if the block uses a library not covered by the data preamble.
123
+
// For example, if normalize used scipy:
124
+
Blockly.Blocks['tidyblocks_transform_normalize'].toplevel_init =
125
+
'from sklearn.preprocessing import MinMaxScaler\n';
126
+
```
127
+
128
+
`BlocklyLayout.getBlocksToplevelInit()` collects all `toplevel_init` strings from
129
+
blocks currently on the canvas, deduplicates them, and prepends them to the generated
130
+
code before execution. This means the import is emitted exactly once regardless of
131
+
how many normalize blocks the user places.
132
+
133
+
Our normalize block only uses pandas, which is already imported by the data block's
134
+
preamble, so **no `toplevel_init` is needed here**.
135
+
136
+
---
137
+
138
+
## Step 4 — Add the block to the toolbox
139
+
140
+
Open `packages/tidyblocks/src/toolbox.ts` and find the `Transform` category.
141
+
Add an entry for the new block:
142
+
143
+
```typescript
144
+
{ kind: 'block', type: 'tidyblocks_transform_normalize' },
145
+
```
146
+
147
+
Place it near related blocks — for example, after `tidyblocks_transform_mutate`:
148
+
149
+
```typescript
150
+
{ kind: 'block', type: 'tidyblocks_transform_mutate' },
151
+
{ kind: 'block', type: 'tidyblocks_transform_normalize' }, // ← new
152
+
```
153
+
154
+
If you want to pre-populate a field value in the toolbox flyout (so dragging the
155
+
block onto the canvas gives a sensible default), use:
156
+
157
+
```typescript
158
+
{
159
+
kind: 'block',
160
+
type: 'tidyblocks_transform_normalize',
161
+
fields: { COLUMN: 'value' }
162
+
},
163
+
```
164
+
165
+
---
166
+
167
+
## Step 5 — Build and verify
168
+
169
+
```bash
170
+
# Rebuild the extension
171
+
npm run build
172
+
173
+
# Start JupyterLab
174
+
jupyter lab
175
+
```
176
+
177
+
1. Open (or create) a `.jpblockly` file.
178
+
2. Select the **Tidy Data** toolbox from the toolbar dropdown.
179
+
3. Expand the **Transform** category — your block should appear.
180
+
4. Drag a data block (e.g. penguins) onto the canvas, then chain the normalize block
181
+
below it.
182
+
5. Click **Run** (▶). The output cell should show the DataFrame with the column
183
+
scaled to 0–1.
184
+
185
+
---
186
+
187
+
## Complete worked example
188
+
189
+
Here is everything together for the normalize block.
190
+
191
+
**`packages/tidyblocks/src/blocks/transform.ts`** — inside `defineBlocksWithJsonArray`:
192
+
193
+
```typescript
194
+
{
195
+
type: 'tidyblocks_transform_normalize',
196
+
message0: 'normalize column %1',
197
+
args0: [
198
+
{ type: 'field_input', name: 'COLUMN', text: 'value' }
199
+
],
200
+
previousStatement: null,
201
+
nextStatement: null,
202
+
colour: '#76AADB',
203
+
tooltip: 'Scale a numeric column to the 0–1 range (min-max normalization).'
204
+
},
205
+
```
206
+
207
+
**`packages/tidyblocks/src/generators/python/transform.ts`**:
208
+
209
+
```typescript
210
+
pythonGenerator.forBlock['tidyblocks_transform_normalize'] = block => {
211
+
const col = block.getFieldValue('COLUMN');
212
+
return (
213
+
`_df = _df.assign(**{'${col}': ` +
214
+
`(_df['${col}'] - _df['${col}'].min()) / ` +
215
+
`(_df['${col}'].max() - _df['${col}'].min())})\n`
216
+
);
217
+
};
218
+
```
219
+
220
+
**`packages/tidyblocks/src/toolbox.ts`**:
221
+
222
+
```typescript
223
+
{ kind: 'block', type: 'tidyblocks_transform_normalize' },
224
+
```
225
+
226
+
---
227
+
228
+
## Adding a new category
229
+
230
+
If your blocks don't fit an existing category, create a new file in `blocks/` and
231
+
`generators/python/`, then:
232
+
233
+
1. Import both files in `packages/tidyblocks/src/index.ts` (the side-effect imports
234
+
section at the top).
235
+
2. Add a new `{ kind: 'category', name: '...', colour: '...', contents: [...] }`
236
+
object to `TIDYBLOCKS_TOOLBOX` in `toolbox.ts`.
237
+
238
+
Choose a colour that contrasts with the existing categories:
239
+
240
+
| Category | Colour |
241
+
|---|---|
242
+
| Data | `#FEBE4C` (amber) |
243
+
| Transform | `#76AADB` (blue) |
244
+
| Combine | `#808080` (grey) |
245
+
| Plot | `#A4C588` (green) |
246
+
| Stats | `#BA93DB` (purple) |
247
+
| Values | `#E7553C` (red-orange) |
248
+
| Operations | `#F9B5B2` (pink) |
249
+
250
+
---
251
+
252
+
## Extending from another JupyterLab plugin
253
+
254
+
You can add blocks from an entirely separate JupyterLab extension without modifying
255
+
this package. Declare `IBlocklyRegistry` as a `required` token in your plugin and
256
+
call the registry methods directly:
257
+
258
+
```typescript
259
+
import { IBlocklyRegistry } from 'jupyter-tidyblocks';
260
+
import * as Blockly from 'blockly';
261
+
import { pythonGenerator } from 'blockly/python';
262
+
263
+
const myPlugin: JupyterFrontEndPlugin<void> = {
264
+
id: 'my-extension:plugin',
265
+
requires: [IBlocklyRegistry],
266
+
activate: (app, registry: IBlocklyRegistry) => {
267
+
// 1. Define the block shape
268
+
registry.registerBlocks([{
269
+
type: 'my_custom_block',
270
+
message0: 'do something with %1',
271
+
args0: [{ type: 'field_input', name: 'VALUE', text: 'x' }],
272
+
previousStatement: null,
273
+
nextStatement: null,
274
+
colour: '#5BA65B',
275
+
tooltip: 'My custom block.'
276
+
}]);
277
+
278
+
// 2. Register the Python generator
279
+
pythonGenerator.forBlock['my_custom_block'] = block => {
280
+
const val = block.getFieldValue('VALUE');
281
+
return `_df = my_transform(_df, '${val}')\n`;
282
+
};
283
+
284
+
// 3. Push the enriched generator back into the registry
285
+
registry.registerGenerator('python', pythonGenerator);
286
+
287
+
// 4. Register a toolbox that includes the new block
288
+
registry.registerToolbox('My Toolbox', {
289
+
kind: 'categoryToolbox',
290
+
contents: [{
291
+
kind: 'category',
292
+
name: 'Custom',
293
+
colour: '#5BA65B',
294
+
contents: [{ kind: 'block', type: 'my_custom_block' }]
295
+
}]
296
+
});
297
+
}
298
+
};
299
+
```
300
+
301
+
> **Important:** Always call `registry.registerGenerator('python', pythonGenerator)`
302
+
> after attaching your `forBlock` handlers. This ensures the registry holds the
303
+
> same generator instance that knows about your new block type, guarding against
304
+
> webpack Module Federation resolving `blockly/python` to a different singleton
305
+
> in a separate bundle.
+11
-14
docs/getting-started.md
+11
-14
docs/getting-started.md
···
38
38
git clone https://github.com/teonbrooks/jupyter-tidyblocks.git
39
39
cd jupyter-tidyblocks
40
40
41
-
# 2. Install Node.js dependencies and build the JS packages
42
-
# (requires Node >= 18 and yarn/jlpm)
43
-
jlpm install
44
-
jlpm build
45
-
46
-
# 3. Install the Python package in editable mode
41
+
# 2. Install the Python package and build the JS packages
47
42
pip install -e ".[dev]"
43
+
jupyter labextension develop . --overwrite
44
+
npm run build
48
45
49
-
# 4. Verify the extension is registered
46
+
# 3. Verify the extension is registered
50
47
jupyter labextension list
51
48
```
52
49
···
154
151
155
152
| Category | Color | Purpose | Example blocks |
156
153
|---|---|---|---|
157
-
| **Data** | Gold | Load a dataset to start a pipeline | Penguins, Colors, CSV, Sequence, User variable |
158
-
| **Transform** | Blue | Reshape or filter `_df` | filter, select, groupby, summarize, sort, bin, sample |
159
-
| **Combine** | Grey | Merge two pipelines | join, glue (concat), cross_join |
154
+
| **Data** | Gold | Load a dataset to start a pipeline | penguins, iris, titanic, gapminder, colors, CSV, sequence, user variable |
155
+
| **Transform** | Blue | Reshape or filter `_df` | filter, select, mutate, arrange, groupby, summarize, count, distinct, bin, slice_head, slice_tail, slice_sample, slice_min, slice_max, relocate |
156
+
| **Combine** | Grey | Merge two pipelines | join, semi_join, anti_join, bind_rows, bind_cols, cross_join |
160
157
| **Plot** | Green | Visualize and terminate a pipeline | scatter, bar, histogram, line, box, violin, heatmap |
161
158
| **Stats** | Purple | Statistical tests and summaries | t-test, k-means, correlation, describe |
162
159
| **Value** | Red | Produce a value (column ref, literal, distribution) | column, number, text, datetime, normal, uniform |
163
-
| **Op** | Pink | Transform a value | arithmetic, compare, logic, ifelse, string, shift |
160
+
| **Op** | Pink | Transform a value | arithmetic, compare, between, logic, ifelse, coalesce, n_distinct, string, shift |
164
161
165
162
> **Pipeline shape**: every pipeline starts with a **Data** block (no left
166
163
> connector), passes through zero or more **Transform** / **Combine** blocks
···
235
232
236
233
```bash
237
234
# Rebuild all packages
238
-
jlpm build
235
+
npm run build
239
236
240
237
# Or watch for changes (rebuilds automatically, then refresh JupyterLab)
241
-
jlpm watch
238
+
npm run watch
242
239
```
243
240
244
241
To run unit tests:
245
242
246
243
```bash
247
-
jlpm test
244
+
npm test
248
245
```
249
246
250
247
To rebuild and reinstall the labextension into JupyterLab's static assets
+19
-4
docs/index.rst
+19
-4
docs/index.rst
···
21
21
22
22
.. toctree::
23
23
:maxdepth: 2
24
-
:caption: Contents:
24
+
:caption: Getting Started:
25
25
26
26
installation
27
27
getting-started
···
39
39
40
40
.. toctree::
41
41
:maxdepth: 2
42
-
:caption: Expanding:
42
+
:caption: Block Reference:
43
+
44
+
blocks-reference
45
+
46
+
.. toctree::
47
+
:maxdepth: 2
48
+
:caption: Extending:
43
49
44
50
other_extensions
51
+
custom-blocks
45
52
46
53
.. toctree::
47
54
:maxdepth: 2
48
55
:caption: Project:
49
56
50
-
jupyterlab-blockly_architecture
51
-
tidyblocks-features
57
+
architecture
58
+
jupyter-tidyblocks-features
52
59
modernization-plan
60
+
work-summary
61
+
62
+
.. toctree::
63
+
:maxdepth: 1
64
+
:caption: Inspirations:
65
+
66
+
inspirations/tidyblocks-features
67
+
inspirations/jupyterlab-blockly_architecture
53
68
54
69
Indices and tables
55
70
==================
+159
docs/inspirations/tidyblocks-features.md
+159
docs/inspirations/tidyblocks-features.md
···
1
+
# tidyblocks — Original Feature Reference
2
+
3
+
**Repository:** https://github.com/gvwilson/tidyblocks (archived August 2024)
4
+
**Author:** Greg Wilson
5
+
**Concept:** A visual, block-based tidy-data analysis environment built on Google Blockly
6
+
with a pipeline execution model, internationalization support (8 languages), and integrated
7
+
plotting and statistics.
8
+
9
+
This document catalogs the features of the original `gvwilson/tidyblocks` project.
10
+
It serves as a reference for what inspired `jupyter-tidyblocks`.
11
+
12
+
---
13
+
14
+
## Execution Model
15
+
16
+
Blocks form **pipelines** — linear chains starting from a data source and flowing through
17
+
transforms into outputs (plots/stats/saves). Each block emits a JSON representation; a
18
+
`Pipeline` runner executes the JSON sequence against an immutable `DataFrame` object.
19
+
Multiple independent pipelines can exist in one workspace.
20
+
21
+
---
22
+
23
+
## Data Source Blocks
24
+
25
+
| Block | Behavior |
26
+
|---|---|
27
+
| `data_colors` | Built-in 11-color RGB dataset |
28
+
| `data_earthquakes` | 2016 earthquake dataset |
29
+
| `data_penguins` | Palmer penguins dataset |
30
+
| `data_phish` | Phish concert dataset |
31
+
| `data_spotify` | Spotify song data |
32
+
| `data_sequence` | Generate 1..N sequence into named column |
33
+
| `data_user` | Reference user-defined dataset by name |
34
+
35
+
---
36
+
37
+
## Transform Blocks
38
+
39
+
| Block | Behavior |
40
+
|---|---|
41
+
| `transform_filter` | Keep rows matching boolean expression |
42
+
| `transform_select` | Keep specified columns |
43
+
| `transform_drop` | Remove specified columns |
44
+
| `transform_create` | Add / replace column with expression |
45
+
| `transform_sort` | Sort by columns; optional descending |
46
+
| `transform_unique` | Deduplicate by columns |
47
+
| `transform_groupBy` | Group rows for subsequent aggregation |
48
+
| `transform_ungroup` | Remove grouping |
49
+
| `transform_summarize` | Aggregate with a summary function |
50
+
| `transform_running` | Cumulative/window operation |
51
+
| `transform_bin` | Discretize numeric column into N buckets |
52
+
| `transform_saveAs` | Persist DataFrame under a name |
53
+
54
+
**Summarize functions:** `all`, `any`, `count`, `max`, `mean`, `median`, `min`, `std`, `sum`, `var`
55
+
56
+
**Running (cumulative) functions:** `cumall`, `cumany`, `cumcount`, `cummax`, `cummean`, `cummin`, `cumsum`
57
+
58
+
---
59
+
60
+
## Combine Blocks
61
+
62
+
| Block | Behavior |
63
+
|---|---|
64
+
| `combine_join` | Inner join on matching column values |
65
+
| `combine_glue` | Vertical stack with source label column |
66
+
67
+
---
68
+
69
+
## Plot Blocks
70
+
71
+
Original plots used a custom JavaScript plotting library.
72
+
73
+
| Block | Behavior |
74
+
|---|---|
75
+
| `plot_bar` | Bar chart (x, y) |
76
+
| `plot_box` | Box plot (x, y) |
77
+
| `plot_dot` | Dot/strip plot (x only) |
78
+
| `plot_histogram` | Histogram (column, nbins) |
79
+
| `plot_scatter` | Scatter (x, y, color, regression toggle) |
80
+
81
+
---
82
+
83
+
## Statistics Blocks
84
+
85
+
| Block | Behavior |
86
+
|---|---|
87
+
| `stats_ttest_one` | One-sample two-sided t-test |
88
+
| `stats_ttest_two` | Two-sample two-sided t-test |
89
+
| `stats_k_means` | K-means clustering |
90
+
| `stats_silhouette` | Silhouette coefficient for cluster quality |
91
+
92
+
---
93
+
94
+
## Operation (Expression) Blocks
95
+
96
+
| Sub-category | Blocks |
97
+
|---|---|
98
+
| Arithmetic | `+`, `-`, `*`, `/`, `%`, `**`, unary `-`, `abs()` |
99
+
| Comparison | `==`, `!=`, `<`, `<=`, `>`, `>=` |
100
+
| Logic | `and`, `or`, `not`, `if/else` ternary |
101
+
| Pairwise extrema | `max(a,b)`, `min(a,b)` |
102
+
| Type checking | `is_missing`, `is_number`, `is_text`, `is_date`, `is_bool` |
103
+
| Type conversion | `to_bool`, `to_datetime`, `to_number`, `to_string` |
104
+
| Datetime extraction | `year`, `month`, `day`, `weekday`, `hour`, `minute`, `second` |
105
+
| Shift (lag/lead) | `shift(col, n)` |
106
+
107
+
---
108
+
109
+
## Value Blocks
110
+
111
+
| Block | Behavior |
112
+
|---|---|
113
+
| `value_column` | Column reference by name |
114
+
| `value_number` | Numeric literal |
115
+
| `value_text` | String literal |
116
+
| `value_logical` | Boolean literal (true/false) |
117
+
| `value_datetime` | Date/time constant (YYYY-MM-DD) |
118
+
| `value_missing` | Explicit NA/NaN value |
119
+
| `value_exponential` | Random draw from Exponential(λ) |
120
+
| `value_normal` | Random draw from Normal(μ, σ) |
121
+
| `value_uniform` | Random draw from Uniform(low, high) |
122
+
123
+
---
124
+
125
+
## Control Blocks
126
+
127
+
| Block | Behavior |
128
+
|---|---|
129
+
| `control_seed` | Set RNG seed for reproducibility |
130
+
131
+
---
132
+
133
+
## Internationalization
134
+
135
+
Block labels available in 8 languages via bundled JSON locale files.
136
+
137
+
---
138
+
139
+
## Key Architectural Concepts
140
+
141
+
### Immutable Pipeline Semantics
142
+
Every transform returns a new DataFrame without mutating the input. Multiple independent
143
+
pipelines can exist in one workspace.
144
+
145
+
### Pipeline Heads
146
+
Blocks with a `hat: 'cap'` property start a pipeline. Multiple pipelines in one workspace
147
+
produce multiple independent execution sequences.
148
+
149
+
### `saveAs` → Named Variables
150
+
`transform_saveAs` persists a DataFrame under a name so later pipelines can reference it
151
+
via `data_user`.
152
+
153
+
### Inline Random Distributions
154
+
`value_normal`, `value_uniform`, `value_exponential` generate random values inline within
155
+
expressions.
156
+
157
+
### `glue` with Source Labels
158
+
`combine_glue` stacks two DataFrames vertically and adds a source-label column, enabling
159
+
before/after or group-comparison workflows.
+35
-50
docs/installation.md
+35
-50
docs/installation.md
···
2
2
3
3
## Requirements
4
4
5
-
- JupyterLab >= 4.0.0
5
+
- Python >= 3.8
6
+
- JupyterLab >= 4.5
6
7
7
8
## Install
8
-
9
-
To install the extension, execute:
10
-
11
-
```bash
12
-
conda install -c conda-forge jupyterlab-blockly
13
-
```
14
-
15
-
or
16
9
17
10
```bash
18
-
pip install jupyterlab-blockly
11
+
pip install jupyter-tidyblocks
19
12
```
20
13
21
-
### Kernels
14
+
### Supported kernels
22
15
23
-
- ipykernel
16
+
- ipykernel (Python)
24
17
- xeus-python
25
18
- xeus-lua
26
-
- [JavaScript](https://github.com/n-riesco/ijavascript#installation)
27
-
- [JavaScript](https://github.com/yunabe/tslab)
19
+
- [ijavascript](https://github.com/n-riesco/ijavascript#installation)
20
+
- [tslab](https://github.com/yunabe/tslab)
28
21
29
22
## Uninstall
30
23
31
-
To remove the extension, execute:
32
-
33
24
```bash
34
-
conda uninstall -c conda-forge jupyterlab-blockly
25
+
pip uninstall jupyter-tidyblocks
35
26
```
36
27
37
-
or
28
+
## Development install
29
+
30
+
**Note:** You will need Node.js >= 18 to build the extension.
38
31
39
32
```bash
40
-
pip uninstall jupyterlab-blockly
41
-
```
33
+
# Create and activate a conda environment
34
+
micromamba create -n tidyblocks -c conda-forge python nodejs pre-commit jupyterlab ipykernel
35
+
micromamba activate tidyblocks
42
36
43
-
## Development install
37
+
# Clone the repo
38
+
git clone https://github.com/teonbrooks/jupyter-tidyblocks
39
+
cd jupyter-tidyblocks
44
40
45
-
**Note:** You will need NodeJS to build the extension package.
41
+
# Install pre-commit hooks
42
+
pre-commit install
46
43
47
-
The `jlpm` command is JupyterLab's pinned version of
48
-
[yarn](https://yarnpkg.com/) that is installed with JupyterLab. You may use
49
-
`yarn` or `npm` in lieu of `jlpm` below.
50
-
51
-
```bash
52
-
micromamba create -n blockly -c conda-forge python nodejs=18 yarn pre-commit jupyterla jupyterlab-language-pack-es-ES jupyterlab-language-pack-fr-FR ipykernel xeus-python xeus-lua
53
-
micromamba activate blockly
54
-
# Clone the repo to your local environment
55
-
# Change directory to the jupyterlab_blockly directory
56
-
# Install package in development mode
44
+
# Install the Python package in editable mode and build the JS packages
57
45
pip install -e ".[dev]"
58
-
# Installing pre-commit to run command when adding commits
59
-
pre-commit install
60
-
# Link your development version of the extension with JupyterLab
61
46
jupyter labextension develop . --overwrite
62
-
# Rebuild extension Typescript source after making changes
63
-
jlpm build
47
+
npm run build
64
48
```
65
49
66
-
You can watch the source directory and run JupyterLab at the same time in different terminals to watch for changes in the extension's source and automatically rebuild the extension.
50
+
You can watch the source directory and run JupyterLab at the same time in
51
+
different terminals to watch for changes and automatically rebuild:
67
52
68
53
```bash
69
-
# Watch the source directory in one terminal, automatically rebuilding when needed
70
-
jlpm watch
71
-
# Run JupyterLab in another terminal
54
+
# Terminal 1 — rebuild on source changes
55
+
npm run watch
56
+
57
+
# Terminal 2 — run JupyterLab
72
58
jupyter lab
73
59
```
74
60
75
-
With the watch command running, every saved change will immediately be built locally and available in your running JupyterLab. Refresh JupyterLab to load the change in your browser (you may need to wait several seconds for the extension to be rebuilt).
61
+
With the watch command running, every saved change will immediately be built
62
+
locally and available in your running JupyterLab. Refresh the browser to load
63
+
the change (you may need to wait several seconds for the rebuild to finish).
76
64
77
-
By default, the `jlpm build` command generates the source maps for this extension to make it easier to debug using the browser dev tools. To also generate source maps for the JupyterLab core extensions, you can run the following command:
65
+
## Development uninstall
78
66
79
67
```bash
80
-
jupyter lab build --minimize=False
68
+
pip uninstall jupyter-tidyblocks
81
69
```
82
70
83
-
## Development uninstall
71
+
Remove the symlink created by `jupyter labextension develop`:
84
72
85
73
```bash
86
-
pip uninstall jupyterlab_blockly
74
+
jupyter labextension list # find the labextensions folder
75
+
# remove the jupyter-tidyblocks-extension symlink from that folder
87
76
```
88
-
89
-
In development mode, you will also need to remove the symlink created by `jupyter labextension develop`
90
-
command. To find its location, you can run `jupyter labextension list` to figure out where the `labextensions`
91
-
folder is located. Then you can remove the symlink named `jupyterlab-blockly` within that folder.
+221
docs/jupyter-tidyblocks-features.md
+221
docs/jupyter-tidyblocks-features.md
···
1
+
# jupyter-tidyblocks Features
2
+
3
+
This document describes all features currently implemented in `jupyter-tidyblocks`.
4
+
5
+
For the blocks that inspired this project, see
6
+
[`inspirations/tidyblocks-features.md`](inspirations/tidyblocks-features.md).
7
+
For the full block reference with generated Python, see
8
+
[`blocks-reference.md`](blocks-reference.md).
9
+
10
+
---
11
+
12
+
## Execution Model
13
+
14
+
Blocks form **pipelines** — linear chains starting from a data source, passing through
15
+
zero or more transforms, and ending at a plot, stats, or display block. The entire
16
+
pipeline generates a single Python code string that executes against the active
17
+
JupyterLab kernel. The running DataFrame is always stored in `_df`.
18
+
19
+
Multiple independent pipelines can exist in one workspace; they execute as one combined
20
+
code cell in top-to-bottom order.
21
+
22
+
---
23
+
24
+
## Data Source Blocks
25
+
26
+
Source blocks start a pipeline. They create `_df` and have no top connector.
27
+
28
+
| Block | Python generated | Notes |
29
+
|---|---|---|
30
+
| `penguins dataset` | `sns.load_dataset('penguins')` | Palmer penguins via seaborn |
31
+
| `iris dataset` | `sns.load_dataset('iris')` | Fisher iris via seaborn |
32
+
| `titanic dataset` | `sns.load_dataset('titanic')` | Titanic survival via seaborn |
33
+
| `gapminder dataset` | `px.data.gapminder()` | GDP/life expectancy via plotly.express |
34
+
| `colors dataset` | `pd.DataFrame({...})` | 11 named colors with RGB values (inline) |
35
+
| `earthquakes dataset` | `pd.read_csv('<url>')` | 2016 global earthquake data |
36
+
| `sequence 1 to N as col` | `pd.DataFrame({'col': range(1, N+1)})` | Integer sequence |
37
+
| `dataset named name` | `name.copy()` | Reference a named DataFrame variable |
38
+
| `read CSV path` | `pd.read_csv('path')` | Load local or remote CSV |
39
+
40
+
---
41
+
42
+
## Transform Blocks
43
+
44
+
Transform blocks read from `_df` and write back to `_df`. Block names follow
45
+
[dplyr](https://dplyr.tidyverse.org/) conventions.
46
+
47
+
| Block | dplyr equivalent | Python generated |
48
+
|---|---|---|
49
+
| `filter where cond` | `filter()` | `_df[cond]` |
50
+
| `select columns` | `select()` | `_df[[cols]]` |
51
+
| `drop columns` | `select(-col)` | `_df.drop(columns=[cols])` |
52
+
| `mutate col = expr` | `mutate()` | `_df.assign(**{'col': expr})` |
53
+
| `rename old → new` | `rename()` | `_df.rename(columns={'old': 'new'})` |
54
+
| `relocate col after` | `relocate()` | column reindex |
55
+
| `arrange by cols` | `arrange()` | `_df.sort_values(by=[cols], ascending=...)` |
56
+
| `distinct by cols` | `distinct()` | `_df.drop_duplicates(subset=[cols])` |
57
+
| `group by cols` | `group_by()` | `_df.groupby([cols])` |
58
+
| `ungroup` | `ungroup()` | `_df.reset_index(drop=True)` |
59
+
| `summarize col = fn` | `summarize()` | `_df.agg({'col': 'fn'})` |
60
+
| `count by cols` | `count()` | `_df.groupby([cols]).size().reset_index(name='n')` |
61
+
| `running fn on col` | — | `_df.assign(col=_df['col'].cumfn())` |
62
+
| `bin col into N` | — | `pd.cut(_df['col'], bins=N)` |
63
+
| `save as name` | — | `name = _df.copy()` |
64
+
| `fill na in col` | — | `_df.fillna({'col': value})` |
65
+
| `drop na from col` | — | `_df.dropna(subset=[cols])` |
66
+
| `slice_sample N rows` | `slice_sample()` | `_df.sample(n=N)` |
67
+
| `slice_head N rows` | `slice_head()` | `_df.head(N)` |
68
+
| `slice_tail N rows` | `slice_tail()` | `_df.tail(N)` |
69
+
| `slice_min N by col` | `slice_min()` | `_df.nsmallest(N, 'col')` |
70
+
| `slice_max N by col` | `slice_max()` | `_df.nlargest(N, 'col')` |
71
+
| `display` | — | `display(_df)` |
72
+
73
+
**Summarize functions:** count, sum, mean, median, min, max, std dev, variance, n distinct, any, all
74
+
75
+
**Running functions:** cumulative sum, cumulative max, cumulative min, cumulative mean, row index
76
+
77
+
---
78
+
79
+
## Combine Blocks
80
+
81
+
Combine blocks merge two DataFrames. The left input is `_df`; the right is a named
82
+
DataFrame (set with **save as**).
83
+
84
+
| Block | dplyr equivalent | Python generated |
85
+
|---|---|---|
86
+
| `join on left = right` | `inner_join()` | `pd.merge(_df, other, left_on=..., right_on=...)` |
87
+
| `semi_join with other on col` | `semi_join()` | `_df[_df['col'].isin(other['col'])]` |
88
+
| `anti_join with other on col` | `anti_join()` | `_df[~_df['col'].isin(other['col'])]` |
89
+
| `bind_rows with other` | `bind_rows()` | `pd.concat([_df, other], ignore_index=True)` |
90
+
| `bind_cols with other` | `bind_cols()` | `pd.concat([_df, other], axis=1)` |
91
+
| `cross join with other` | — | `_df.merge(other, how='cross')` |
92
+
93
+
---
94
+
95
+
## Plot Blocks
96
+
97
+
Plot blocks visualize `_df` using [plotly.express](https://plotly.com/python/plotly-express/).
98
+
They are terminal blocks (left connector only; nothing chains below them).
99
+
100
+
| Block | Python generated |
101
+
|---|---|
102
+
| `bar chart x y` | `px.bar(_df, x='x', y='y').show()` |
103
+
| `box plot x y` | `px.box(_df, x='x', y='y').show()` |
104
+
| `dot plot x` | `px.strip(_df, x='x').show()` |
105
+
| `histogram x bins` | `px.histogram(_df, x='x', nbins=N).show()` |
106
+
| `scatter x y color` | `px.scatter(_df, x='x', y='y', color='c').show()` |
107
+
| `line chart x y color` | `px.line(_df, x='x', y='y', color='c').show()` |
108
+
| `violin x y` | `px.violin(_df, x='x', y='y').show()` |
109
+
| `heatmap` | `px.imshow(_df.corr()).show()` |
110
+
111
+
---
112
+
113
+
## Stats Blocks
114
+
115
+
Stats blocks run statistical analyses on `_df` using scipy and scikit-learn.
116
+
They are terminal blocks.
117
+
118
+
| Block | Python generated |
119
+
|---|---|
120
+
| `one-sample t-test col vs mean` | `scipy.stats.ttest_1samp(_df['col'].dropna(), mean)` |
121
+
| `two-sample t-test col by group` | `scipy.stats.ttest_ind(group1, group2)` |
122
+
| `k-means on cols k clusters` | `KMeans(n_clusters=k).fit(_df[[cols]])` |
123
+
| `silhouette score on cols` | `silhouette_score(_df[[cols]], labels)` |
124
+
| `correlation of cols` | `_df[[cols]].corr()` |
125
+
| `describe` | `_df.describe()` |
126
+
127
+
---
128
+
129
+
## Value Blocks
130
+
131
+
Value blocks produce a single value for use inside filter conditions or mutate
132
+
expressions. They have no top/bottom connectors — they attach to input slots.
133
+
134
+
| Block | Python generated |
135
+
|---|---|
136
+
| `column name` | `_df['name']` |
137
+
| `number N` | `N` |
138
+
| `text "str"` | `'str'` |
139
+
| `true / false` | `True` / `False` |
140
+
| `datetime YYYY-MM-DD` | `pd.Timestamp('YYYY-MM-DD')` |
141
+
| `missing` | `None` |
142
+
| `normal(μ, σ)` | `np.random.normal(μ, σ, len(_df))` |
143
+
| `uniform(low, high)` | `np.random.uniform(low, high, len(_df))` |
144
+
| `exponential(λ)` | `np.random.exponential(λ, len(_df))` |
145
+
146
+
---
147
+
148
+
## Operation Blocks
149
+
150
+
Operation blocks build expressions used inside filter/mutate/etc.
151
+
152
+
| Sub-category | Blocks | dplyr equivalent |
153
+
|---|---|---|
154
+
| Arithmetic | `+`, `-`, `*`, `/`, `%`, `**`, unary `-`, `abs()` | — |
155
+
| Comparison | `==`, `!=`, `<`, `<=`, `>`, `>=` | — |
156
+
| Between | `col between lo and hi` | `between()` |
157
+
| Logic | `and`, `or`, `not` | — |
158
+
| If/else | `if cond then a else b` | `if_else()` |
159
+
| Coalesce | `coalesce(col, default)` | `coalesce()` |
160
+
| N distinct | `n_distinct(col)` | `n_distinct()` |
161
+
| Type checking | `is missing`, `is number`, `is text`, `is date`, `is bool` | — |
162
+
| Type conversion | `to bool`, `to datetime`, `to number`, `to string` | — |
163
+
| Datetime extraction | `year`, `month`, `day`, `weekday`, `hour`, `minute`, `second` | — |
164
+
| Shift | `shift col by n` | `lag()` / `lead()` |
165
+
| Math | `round`, `floor`, `ceil`, `log`, `sqrt`, `exp` | — |
166
+
| String | `upper`, `lower`, `strip` | — |
167
+
| String contains | `col contains pattern` | `str_detect()` |
168
+
169
+
---
170
+
171
+
## Kernel Support
172
+
173
+
The same workspace can generate code for multiple kernel languages. The active kernel
174
+
is selected from the **Kernel** dropdown in the editor toolbar.
175
+
176
+
| Kernel | Language | Generator |
177
+
|---|---|---|
178
+
| ipykernel / xeus-python | Python | `blockly/python` |
179
+
| ijavascript / tslab | JavaScript | `blockly/javascript` |
180
+
| xeus-lua | Lua | `blockly/lua` |
181
+
182
+
---
183
+
184
+
## Extensibility
185
+
186
+
The `IBlocklyRegistry` token (provided by `jupyter-tidyblocks`) allows any JupyterLab
187
+
plugin to:
188
+
189
+
- Register new block shapes (`registerBlocks`)
190
+
- Register new toolboxes (`registerToolbox`)
191
+
- Register new code generators (`registerGenerator`)
192
+
193
+
See [Adding Custom Blocks](custom-blocks.md) for a step-by-step guide.
194
+
195
+
---
196
+
197
+
## File Format
198
+
199
+
Workspace state is saved as `.jpblockly` files — JSON containing the serialized Blockly
200
+
workspace state. Files can be committed to version control and reopened to restore the
201
+
exact block arrangement.
202
+
203
+
---
204
+
205
+
## Build System
206
+
207
+
- **Turborepo** for monorepo task orchestration across three packages
208
+
- **Vite** (library mode) for `packages/blockly` and `packages/tidyblocks`
209
+
- **`@jupyterlab/builder`** (webpack) for `packages/blockly-extension`
210
+
- **npm workspaces** with `overrides` for dependency version pinning
211
+
- **hatchling + hatch-jupyter-builder** for the Python wheel build
212
+
213
+
---
214
+
215
+
## Python Runtime Dependencies
216
+
217
+
Generated code uses the following libraries (install separately):
218
+
219
+
```bash
220
+
pip install pandas numpy seaborn plotly scipy scikit-learn
221
+
```
docs/jupyterlab-blockly_architecture.md
docs/inspirations/jupyterlab-blockly_architecture.md
docs/jupyterlab-blockly_architecture.md
docs/inspirations/jupyterlab-blockly_architecture.md
+87
-120
docs/other_extensions.md
+87
-120
docs/other_extensions.md
···
1
-
# Other extensions
1
+
# Extending jupyter-tidyblocks
2
2
3
-
The JupyterLab-Blockly extension is ready to be used as a base for other projects: you can register new Blocks, Toolboxes and Generators. It is a great tool for fast prototyping.
3
+
The `jupyter-tidyblocks` core extension exposes an `IBlocklyRegistry` token that
4
+
other JupyterLab plugins can use to register new blocks, toolboxes, and generators —
5
+
without modifying this repository.
4
6
5
-
## Creating a new JupyterLab extension
6
-
You can easily create a new JupyterLab extension by using the official `copier` template, documented [here](https://github.com/jupyterlab/extension-template).
7
-
8
-
After installing the needed plugins (mentioned in the above link) and creating an extension directory, you can run the following command:
9
-
```
10
-
copier copy --trust https://github.com/jupyterlab/extension-template .
11
-
```
12
-
which will ask you to fill some basic information about your project. Once completed, the directory will be populated with several files, all forming the base of your project. You will mostly work in the `index.ts` file, located in the `src` folder.
7
+
For a complete step-by-step walkthrough of adding blocks and toolboxes, see
8
+
[**Adding Custom Blocks**](custom-blocks.md).
13
9
14
-
An example of creating a simple JupyterLab extension, which also contains the instructions of how to fill the information asked by the `copier` template, can be found [here](https://github.com/jupyterlab/extension-examples/tree/master/hello-world).
10
+
---
15
11
12
+
## Quick start
16
13
17
-
## Importing JupyterLab-Blockly
18
-
Firstly you need to install and add `jupyterlab-blockly` as a dependency for your extension:
19
-
```
20
-
jlpm add jupyterlab-blockly
21
-
```
14
+
### 1. Create a new JupyterLab extension
22
15
23
-
Once it is part of your project, all you need to do is import `IBlocklyRegistry`, as it follows:
24
-
```typescript
25
-
// src/index.ts
16
+
Use the official copier template:
26
17
27
-
import { IBlocklyRegistry } from 'jupyterlab-blockly';
18
+
```bash
19
+
pip install copier jinja2-time
20
+
copier copy --trust https://github.com/jupyterlab/extension-template .
28
21
```
29
22
30
-
The `BlocklyRegistry` is the class that the JupyterLab-Blockly extension exposes to other plugins. This registry allows other plugins to register new Toolboxes, Blocks and Generators that users can use in the Blockly editor.
31
-
32
-
### Registering new Blocks
33
-
The `IBlocklyRegistry` offers a function `registerBlocks`, which allows you to include new Blocks in your project. Blockly offers a [tool](https://blockly-demo.appspot.com/static/demos/blockfactory/index.html) which helps you easily create new Blocks and get their JSON definition and generator code in all supported programming languages.
23
+
Fill in the prompts. You will mostly work in `src/index.ts`.
34
24
35
-
**NOTE** : Once you create a new block, it won't appear into your Blockly editor, unless you add it to a Toolbox.
25
+
### 2. Add `jupyter-tidyblocks` as a dependency
36
26
37
-
```typescript
38
-
/**
39
-
* Register new blocks.
40
-
*
41
-
* @argument blocks Blocks to register.
42
-
*/
43
-
registerBlocks(blocks: BlockDefinition[]): void {
44
-
Blockly.defineBlocksWithJsonArray(blocks);
45
-
}
27
+
```bash
28
+
npm install jupyter-tidyblocks
46
29
```
47
30
48
-
### Registering a new Toolbox
49
-
Using the `registerToolbox` function, provided by `IBlocklyRegistry`, you can register a new toolbox. Once registered, the toolbox will appear automatically in your Blockly editor. You can find more information about switching to another toolbox [here](https://jupyterlab-blockly.readthedocs.io/en/latest/toolbox.html).
31
+
Add it to your extension's `package.json` dependencies and declare it as a
32
+
shared singleton in the `jupyterlab.sharedPackages` section so both your
33
+
extension and the core share the same registry instance:
50
34
51
-
```typescript
52
-
/**
53
-
* Register a toolbox for the editor.
54
-
*
55
-
* @argument name Name of the toolbox.
56
-
*
57
-
* @argument value Toolbox to register.
58
-
*/
59
-
registerToolbox(name: string, value: ToolboxDefinition): void {
60
-
this._toolboxes.set(name, value);
35
+
```json
36
+
"jupyterlab": {
37
+
"sharedPackages": {
38
+
"jupyter-tidyblocks": {
39
+
"bundled": false,
40
+
"singleton": true
41
+
}
61
42
}
43
+
}
62
44
```
63
45
64
-
### Registering a new Generator
65
-
Lastly, `IBlocklyRegistry` offers the function `registerGenerator` which lets you register a new Generator. You can read more about switching kernels [here](https://jupyterlab-blockly.readthedocs.io/en/latest/kernels.html).
46
+
### 3. Use `IBlocklyRegistry` in your plugin
66
47
67
48
```typescript
49
+
// src/index.ts
50
+
import { IBlocklyRegistry } from 'jupyter-tidyblocks';
51
+
import * as Blockly from 'blockly';
52
+
import { pythonGenerator } from 'blockly/python';
68
53
69
-
/**
70
-
* Register new generators.
71
-
*
72
-
* @argument name Name of the generator.
73
-
*
74
-
* @argument generator Generator to register.
75
-
*
76
-
* #### Notes
77
-
* When registering a generator, the name should correspond to the language
78
-
* used by a kernel.
79
-
*
80
-
* If you register a generator for an existing language this will be overwritten.
81
-
*/
82
-
registerGenerator(name: string, generator: Blockly.Generator): void {
83
-
this._generators.set(name, generator);
84
-
}
85
-
```
54
+
const plugin: JupyterFrontEndPlugin<void> = {
55
+
id: 'my-extension:plugin',
56
+
autoStart: true,
57
+
requires: [IBlocklyRegistry],
58
+
activate: (app, registry: IBlocklyRegistry) => {
86
59
87
-
88
-
## Example - JupyterLab-Niryo-One
89
-
The [JupyterLab-Niryo-One](https://github.com/QuantStack/jupyterlab-niryo-one/) extension was built on top of JupyterLab-Blockly and poses as the perfect example. The [Github repository](https://github.com/QuantStack/jupyterlab-niryo-one/) gives access to its entire codebase.
60
+
// Register block shape(s)
61
+
registry.registerBlocks([{
62
+
type: 'my_custom_block',
63
+
message0: 'do something with %1',
64
+
args0: [{ type: 'field_input', name: 'VALUE', text: 'x' }],
65
+
previousStatement: null,
66
+
nextStatement: null,
67
+
colour: '#5BA65B',
68
+
tooltip: 'My custom block.'
69
+
}]);
90
70
91
-
The following code snippet showcases how to register a new toolbox, `BlocklyNiryo.Toolbox`, as `niryo`.
92
-
```typescript
93
-
// src/index.ts : 10-23
71
+
// Register the Python generator
72
+
pythonGenerator.forBlock['my_custom_block'] = block => {
73
+
const val = block.getFieldValue('VALUE');
74
+
return `_df = my_transform(_df, '${val}')\n`;
75
+
};
94
76
95
-
/**
96
-
* Initialization data for the jupyterlab-niryo-one extension.
97
-
*/
98
-
const plugin: JupyterFrontEndPlugin<void> = {
99
-
id: 'jupyterlab-niryo-one:plugin',
100
-
autoStart: true,
101
-
requires: [IBlocklyRegistry],
102
-
activate: (app: JupyterFrontEnd, blockly: IBlocklyRegistry) => {
103
-
console.log('JupyterLab extension jupyterlab-niryo-one is activated!');
77
+
// Push the enriched generator back into the registry
78
+
registry.registerGenerator('python', pythonGenerator);
104
79
105
-
//Registering the new toolbox containing all Niryo One blocks.
106
-
blockly.registerToolbox('niryo', BlocklyNiryo.Toolbox);
80
+
// Register a toolbox containing the new block
81
+
registry.registerToolbox('My Toolbox', {
82
+
kind: 'categoryToolbox',
83
+
contents: [{
84
+
kind: 'category',
85
+
name: 'Custom',
86
+
colour: '#5BA65B',
87
+
contents: [{ kind: 'block', type: 'my_custom_block' }]
88
+
}]
89
+
});
107
90
}
108
91
};
109
92
```
110
93
111
-
**NOTE** : `BlocklyNiryo` is defined in `niryo-one-python-generators.ts`.
94
+
> **Important:** Always call `registry.registerGenerator('python', pythonGenerator)`
95
+
> after attaching `forBlock` handlers. This guards against webpack Module Federation
96
+
> resolving `blockly/python` to a different singleton across bundles.
112
97
98
+
---
113
99
114
-
## Additional configurations
100
+
## `IBlocklyRegistry` API
101
+
102
+
| Method | Description |
103
+
|---|---|
104
+
| `registerBlocks(blocks)` | Register block shape definitions (same as `Blockly.defineBlocksWithJsonArray`) |
105
+
| `registerToolbox(name, toolbox)` | Add a named toolbox that appears in the editor toolbar dropdown |
106
+
| `registerGenerator(language, generator)` | Replace or add a code generator for a kernel language |
107
+
| `toolboxes` | Read-only `Map<string, ToolboxDefinition>` of all registered toolboxes |
108
+
| `generators` | Read-only `Map<string, Blockly.Generator>` of all registered generators |
115
109
116
-
You will need to request the `jupyterlab-blockly` package as a dependency for your extension, in order to ensure it is installed and available to provide the token `IBlocklyRegistry`. To do this, you need to add the following line to your `pyproject.toml` file.
110
+
---
117
111
118
-
```
119
-
// pyproject.toml : 26
112
+
## Add to `pyproject.toml`
120
113
114
+
Declare `jupyter-tidyblocks` as a Python dependency so pip installs it alongside
115
+
your extension:
116
+
117
+
```toml
118
+
[project]
121
119
dependencies = [
122
-
"jupyterlab-blockly>=0.3.2,<0.4",
123
-
... // add any additional dependencies needed for your extension
120
+
"jupyter-tidyblocks>=0.1.0",
121
+
# ... other dependencies
124
122
]
125
123
```
126
-
127
-
Additionally, you will need to add the webpack configuration for loading the `Blockly` source maps. You can do this, by creating the following `webpack.config.js` file inside your root directory:
128
-
129
-
```js
130
-
// @ts-check
131
-
132
-
module.exports = /** @type { import('webpack').Configuration } */ ({
133
-
devtool: 'source-map',
134
-
module: {
135
-
rules: [
136
-
// Load Blockly source maps.
137
-
{
138
-
test: /(blockly\/.*\.js)$/,
139
-
use: [require.resolve('source-map-loader')],
140
-
enforce: 'pre'
141
-
}
142
-
].filter(Boolean)
143
-
},
144
-
// https://github.com/google/blockly-samples/blob/9974e85becaa8ad17e35b588b95391c85865dafd/plugins/dev-scripts/config/webpack.config.js#L118-L120
145
-
ignoreWarnings: [/Failed to parse source map/]
146
-
});
147
-
```
148
-
149
-
and by connecting the `webpack` config to your `jupyterlab` instance, which entails adding the following line inside your `package.json`:
150
-
151
-
```json
152
-
"jupyterlab": {
153
-
...
154
-
"webpackConfig": "./webpack.config.js"
155
-
}
156
-
```
+30
-4
docs/toolbox.md
+30
-4
docs/toolbox.md
···
1
1
# Toolbox
2
2
3
-
The toolbox, a main element of the Blockly editor, is situated on the left side of the screen. It encompasses all available blocks, organized in categories for easier access.
3
+
The toolbox is the panel on the left side of the Blockly editor. It contains all
4
+
available blocks organized into categories.
4
5
5
6
<p align="center">
6
7
<img src="_static/toolboxView.gif" alt="Toolbox View"/>
7
8
</p>
8
9
9
-
## Switching to another toolbox
10
+
## Tidy Data toolbox
10
11
11
-
If you have installed or created another extension, on top of the JupyterLab-Blockly extension, which includes a new tooolbox, you can switch to it by simply pressing the drop down menu on the upper-right corner.
12
+
When a `.jpblockly` file is opened with the **Tidy Data** toolbox selected (the
13
+
default), the sidebar shows seven categories:
14
+
15
+
| Category | Colour | Purpose |
16
+
|---|---|---|
17
+
| **Data** | Gold `#FEBE4C` | Source blocks that load a dataset into `_df` |
18
+
| **Transform** | Blue `#76AADB` | Row/column operations on `_df` |
19
+
| **Combine** | Grey `#808080` | Multi-table joins and stacks |
20
+
| **Plot** | Green `#A4C588` | Visualizations that render `_df` |
21
+
| **Stats** | Purple `#BA93DB` | Statistical tests and summaries |
22
+
| **Values** | Red `#E7553C` | Literal values and column references |
23
+
| **Operations** | Pink `#F9B5B2` | Expressions (arithmetic, logic, string, …) |
24
+
25
+
For a full list of every block in each category, see
26
+
[Block Reference](blocks-reference.md).
27
+
28
+
## Switching toolboxes
29
+
30
+
If another extension has registered an additional toolbox, you can switch to it
31
+
using the **Toolbox** dropdown in the editor toolbar (upper-right area of the
32
+
editor panel).
12
33
13
34
14
35
15
-
**NOTE** : The toolbox `niryo` from the image above is part of the JupyterLab-Niryo-One extension, which is built on top of the JupyterLab-Blockly extesnion and is meant to offer blocks which can control the Niryo One robot. You can read more about it on its [Github repository](https://github.com/QuantStack/jupyterlab-niryo-one).
36
+
## Adding your own toolbox
37
+
38
+
You can register a custom toolbox from a JupyterLab plugin using
39
+
`IBlocklyRegistry.registerToolbox(name, definition)`. Once registered it appears
40
+
automatically in the dropdown. See [Extending jupyter-tidyblocks](other_extensions.md)
41
+
and [Adding Custom Blocks](custom-blocks.md) for a full guide.