> ## Documentation Index
> Fetch the complete documentation index at: https://aegean.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Rerun Lakehouse Visualizer, Flickr8k Demo

> Interactive multimodal visualization of Flickr8k images and captions from the Auraison lakehouse using the Rerun SDK web viewer.

Interactive multimodal visualization of the **Flickr8k** dataset stored in the
Auraison lakehouse (`s3://landing/flickr-8k/`), powered by the
[Rerun SDK](https://github.com/rerun-io/rerun).

**What this demo shows:**

* Loading image + caption pairs directly from MinIO S3 via `boto3`
* Logging them to Rerun with a sequential timeline (`image_idx`)
* Saving a portable `.rrd` recording that can be shared or embedded
* Embedding the Rerun web viewer in a Mintlify MDX page via `<iframe>`

**Prerequisites:**

```bash theme={null}
cd data-plane
# External TrueNAS infra, set .env with MINIO_ENDPOINT, MINIO_ACCESS_KEY, MINIO_SECRET_KEY
# Or start local infra:
# docker compose --profile local-infra up -d
```

```python theme={null}
import os, sys
# Ensure data-plane root is on the path
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath('.')), ''))

S3_ENDPOINT = os.getenv('MINIO_ENDPOINT', 'http://localhost:29000')
S3_KEY      = os.getenv('MINIO_ACCESS_KEY', 'minio')
S3_SECRET   = os.getenv('MINIO_SECRET_KEY', 'minio123')

BUCKET   = 'landing'
PREFIX   = 'flickr-8k'
N_IMAGES = 20          # images to stream into Rerun
RRD_PATH = '/tmp/flickr8k_demo.rrd'

print(f'MinIO endpoint : {S3_ENDPOINT}')
print(f'Dataset prefix : s3://{BUCKET}/{PREFIX}/')
print(f'Images to load : {N_IMAGES}')
print(f'RRD output     : {RRD_PATH}')
```

```output theme={null}
MinIO endpoint : http://192.168.1.26:9000
Dataset prefix : s3://landing/flickr-8k/
Images to load : 20
RRD output     : /tmp/flickr8k_demo.rrd
```

## Step 1, Load Captions from S3

Flickr8k caption file format: CSV with `image,caption` columns.
Each image has 5 reference captions from different annotators.

```python theme={null}
import boto3
from botocore.config import Config
from collections import defaultdict

s3 = boto3.client(
    's3',
    endpoint_url=S3_ENDPOINT,
    aws_access_key_id=S3_KEY,
    aws_secret_access_key=S3_SECRET,
    config=Config(signature_version='s3v4'),
)

# Fetch captions index
obj = s3.get_object(Bucket=BUCKET, Key=f'{PREFIX}/text/captions.txt')
lines = obj['Body'].read().decode('utf-8').splitlines()

captions = defaultdict(list)
for line in lines[1:]:          # skip header
    parts = line.split(',', 1)
    if len(parts) == 2:
        fname, cap = parts
        captions[fname.strip()].append(cap.strip())
    if len(captions) >= N_IMAGES:
        break

filenames = list(captions.keys())[:N_IMAGES]

print(f'Caption file: {len(lines)-1} entries total')
print(f'Unique images selected: {len(filenames)}')
print(f'\nSample, {filenames[0]}:')
for i, cap in enumerate(captions[filenames[0]]):
    print(f'  {i+1}. {cap}')
```

```output theme={null}
Caption file: 40455 entries total
Unique images selected: 20

Sample, 1000268201_693b08cb0e.jpg:
  1. A child in a pink dress is climbing up a set of stairs in an entry way .
  2. A girl going into a wooden building .
  3. A little girl climbing into a wooden playhouse .
  4. A little girl climbing the stairs to her playhouse .
  5. A little girl in a pink dress going into a wooden cabin .
```

## Step 2, Log Images + Captions to Rerun

We initialise a Rerun **recording stream** and save it to a `.rrd` file.
Each image is logged as `rr.EncodedImage` (JPEG bytes, no decode needed)
and each set of captions as `rr.TextDocument` with Markdown formatting.

Timeline: `image_idx`, a sequence timeline, one step per image.

```python theme={null}
import rerun as rr
import rerun.blueprint as rrb

# Initialise recording (headless, no viewer spawned)
rr.init('auraison/flickr8k', spawn=False)
rr.save(RRD_PATH)

# Blueprint: image on left (3/5 width), captions on right (2/5)
blueprint = rrb.Horizontal(
    rrb.Spatial2DView(name='Image', origin='/image'),
    rrb.TextDocumentView(name='Captions', origin='/captions'),
    column_shares=[3, 2],
)
rr.send_blueprint(blueprint)

print(f'Streaming {len(filenames)} images → {RRD_PATH} …\n')

for idx, fname in enumerate(filenames):
    img_bytes = s3.get_object(
        Bucket=BUCKET, Key=f'{PREFIX}/images/{fname}'
    )['Body'].read()

    rr.set_time('image_idx', sequence=idx)

    # JPEG-encoded image, no PIL decode needed
    rr.log('/image', rr.EncodedImage(contents=img_bytes, media_type='image/jpeg'))

    # All 5 captions as a Markdown document
    caption_md = f'**{fname}**\n\n' + '\n\n'.join(
        f'{i+1}. {c}' for i, c in enumerate(captions[fname])
    )
    rr.log('/captions', rr.TextDocument(caption_md, media_type=rr.MediaType.MARKDOWN))

    size_kb = len(img_bytes) / 1024
    print(f'  [{idx+1:2d}/{len(filenames)}]  {fname}  ({size_kb:.0f} KiB)')

print(f'\n✓ Recording saved to {RRD_PATH}')
import os
rrd_size_kb = os.path.getsize(RRD_PATH) / 1024
print(f'  File size: {rrd_size_kb:.0f} KiB')
```

```output theme={null}
Streaming 20 images → /tmp/flickr8k_demo.rrd …

  [ 1/20]  1000268201_693b08cb0e.jpg  (195 KiB)
  [ 2/20]  1001773457_577c3a7d70.jpg  (139 KiB)
  [ 3/20]  1002674143_1b742ab4b8.jpg  (156 KiB)
  [ 4/20]  1003163366_44323f5815.jpg  (150 KiB)
  [ 5/20]  1007129816_e794419615.jpg  (118 KiB)
  [ 6/20]  1007320043_627395c3d8.jpg  (175 KiB)
  [ 7/20]  1009434119_febe49276a.jpg  (166 KiB)
  [ 8/20]  1012212859_01547e3f17.jpg  (124 KiB)
  [ 9/20]  1015118661_980735411b.jpg  (157 KiB)
  [10/20]  1015584366_dfcec3c85a.jpg  (145 KiB)
  [11/20]  101654506_8eb26cfb60.jpg  (99 KiB)
  [12/20]  101669240_b2d3e7f17b.jpg  (161 KiB)
  [13/20]  1016887272_03199f49c4.jpg  (115 KiB)
  [14/20]  1019077836_6fc9b15408.jpg  (251 KiB)
  [15/20]  1019604187_d087bf9a5f.jpg  (110 KiB)
  [16/20]  1020651753_06077ec457.jpg  (161 KiB)
  [17/20]  1022454332_6af2c1449a.jpg  (128 KiB)
  [18/20]  1022454428_b6b660a67b.jpg  (110 KiB)
  [19/20]  1022975728_75515238d8.jpg  (101 KiB)
  [20/20]  102351840_323e3de834.jpg  (48 KiB)

✓ Recording saved to /tmp/flickr8k_demo.rrd
  File size: 2175 KiB
```

## Step 3, Open in the Rerun Web Viewer

To view the recording locally, run:

```bash theme={null}
# Serve via gRPC + open browser automatically
uv run python scripts/rerun_demo.py --count 20

# Or open an existing .rrd file with the native viewer
rerun /tmp/flickr8k_demo.rrd
```

The recording can also be embedded in any web page using the hosted Rerun viewer at
`https://app.rerun.io`, see the **Mintlify Embed** section below.

### What Rerun shows

| Panel                 | Content                               |
| --------------------- | ------------------------------------- |
| Left (Spatial 2D)     | JPEG image, pan, zoom, inspect pixels |
| Right (Text Document) | 5 reference captions in Markdown      |
| Bottom timeline       | `image_idx`, scrub to navigate images |

## Step 4, Mintlify Embed (iframe)

The Rerun web viewer at `https://app.rerun.io` can load any publicly hosted `.rrd` file
via its `url` query parameter. The embed below uses Rerun's own ARKit Scenes example to
validate that the Mintlify `<iframe>` component renders correctly:

```mdx theme={null}
<iframe
  src="https://app.rerun.io/version/0.29.2/index.html?url=https://app.rerun.io/version/0.29.2/examples/arkit_scenes.rrd"
  width="100%"
  height="600px"
  style={{ border: 'none', borderRadius: '8px' }}
/>
```

To embed the **Flickr8k** recording instead, upload `flickr8k_demo.rrd` to a public URL
(e.g. Cloudflare R2, GitHub Releases, or MinIO with public bucket policy) and substitute
the `url` parameter.

### Live Embed Test

The iframe below loads the Rerun web viewer with the ARKit Scenes example recording to
verify that the Mintlify `<iframe>` component renders correctly:

<iframe src="https://app.rerun.io/version/0.29.2/index.html?url=https://app.rerun.io/version/0.29.2/examples/arkit_scenes.rrd" width="100%" height="640px" style={{ border: 'none', borderRadius: '8px' }} />

```python theme={null}
print('=== Rerun Lakehouse Visualizer, Summary ===')
print(f'Dataset        : Flickr8k  (landing/flickr-8k/)')
print(f'Images logged  : {len(filenames)}')
print(f'Captions/image : 5 reference captions each')
print(f'Recording file : {RRD_PATH}  ({rrd_size_kb:.0f} KiB)')
print()
print('Entity tree logged to Rerun:')
print('  /image    → rr.EncodedImage (JPEG)')
print('  /captions → rr.TextDocument (Markdown)')
print()
print('Timeline: image_idx (sequence, 0 …', len(filenames)-1, ')')
print()
print('To serve live:')
print('  uv run python scripts/rerun_demo.py --count 20')
print('  → http://localhost:9090')
```

```output theme={null}
=== Rerun Lakehouse Visualizer, Summary ===
Dataset        : Flickr8k  (landing/flickr-8k/)
Images logged  : 20
Captions/image : 5 reference captions each
Recording file : /tmp/flickr8k_demo.rrd  (2175 KiB)

Entity tree logged to Rerun:
  /image    → rr.EncodedImage (JPEG)
  /captions → rr.TextDocument (Markdown)

Timeline: image_idx (sequence, 0 … 19 )

To serve live:
  uv run python scripts/rerun_demo.py --count 20
  → http://localhost:9090
```

***

<Callout icon="pen-to-square" iconType="regular">
  [Edit this page on GitHub](https://github.com/aegean-ai/eaia/edit/main/src/products/tech-demonstrators/lakehouse/rerun-flickr8k/index.mdx) or [file an issue](https://github.com/aegean-ai/eaia/issues/new/choose).
</Callout>