1. Extracting the Raw Metadata

Creating a complete, unfiltered snapshot of your MIDI library

The first step in normalizing a MIDI library is to capture everything we can possibly know about each file before any cleanup or renaming occurs. This is the “archaeological dig” phase: we gather the raw material that all later steps depend on.

A dedicated metadata‑dump script scans the target folder and extracts:

  • the current filename
  • any embedded title information
  • any embedded composer information
  • comments or text events
  • tempo markings
  • time signatures
  • key signatures
  • other recoverable metadata from Track 0 or additional tracks

The script writes all of this into a single raw metadata spreadsheet (XLSX).
This file is intentionally unedited and uncorrected — it reflects the true state of the library, including inconsistencies, encoding issues, and missing data.

Why this step matters

1. It preserves the original state of the library

Before any normalization or renaming happens, this spreadsheet becomes a permanent record of:

  • what the filenames were
  • what metadata was present
  • what metadata was missing
  • how the files were structured internally

If anything ever needs to be undone, re‑checked, or re‑interpreted, this file is the fallback.

2. It provides the raw material for normalization

Later scripts rely on this dump to:

  • repair encoding issues
  • standardize title and composer fields
  • extract primary composers
  • detect malformed metadata
  • build a consistent metadata table

Without this raw snapshot, normalization would be guesswork.

3. It separates extraction from interpretation

This is a key design principle of the workflow.

Extraction is mechanical.
Interpretation is human.
Normalization is algorithmic.
Renaming is deterministic.

By keeping these phases separate, the pipeline stays transparent and reversible.

What the output looks like

The raw metadata file is an XLSX spreadsheet with one row per MIDI file and columns such as:

  • filename
  • title_raw
  • composer_raw
  • comments_raw
  • tempo
  • time_signature
  • key_signature

The exact column names may vary depending on the script version, but the goal is always the same: capture everything without altering it.

How this file is used later

The raw metadata file is:

  • a reference archive
  • a safety net
  • the input to the metadata normalizer
  • a historical snapshot of the library before cleanup

The normalized metadata file (created in the next step) uses this raw dump as its source, but the raw file itself is never overwritten. It remains a trustworthy record of the original state.

Summary

Extracting the raw metadata is the foundation of the entire normalization workflow.
It creates a complete, unfiltered, archival‑grade snapshot of the MIDI library before any cleanup occurs. This ensures that every later step — normalization, human review, and renaming — is based on accurate, preserved information and can be repeated or reversed at any time.

This script performs a complete, unfiltered scan of every MIDI file in the folder and extracts all recoverable metadata. The output is saved as both raw_metadata.xlsx and raw_metadata.csv.

This file is never edited and never overwritten — it serves as the archival snapshot of the library before normalization.

 

import csv
import re
from pathlib import Path
from openpyxl import Workbook
import mido

# Folder containing the MIDI files to scan
MIDI_FOLDER = Path(".")

# Output file (raw, unnormalized metadata)
OUTPUT_XLSX = "raw_metadata.xlsx"
OUTPUT_CSV = "raw_metadata.csv"


# ------------------------------------------------------------
# Encoding helpers
# ------------------------------------------------------------

def fix_encoding(s):
    """Attempt to repair common CP1252/UTF‑8 mojibake."""
    if not isinstance(s, str):
        return ""
    try:
        return s.encode("cp1252").decode("utf-8")
    except:
        return s


# ------------------------------------------------------------
# Metadata extraction helpers
# ------------------------------------------------------------

def extract_meta_from_track(track):
    """Extract title, composer, comments, tempo, key, and time signature."""
    title = ""
    composer = ""
    comments = []
    tempo = ""
    time_sig = ""
    key_sig = ""

    for msg in track:
        if msg.type == "track_name" and not title:
            title = msg.name
        elif msg.type == "text":
            comments.append(msg.text)
        elif msg.type == "copyright" and not composer:
            composer = msg.text
        elif msg.type == "set_tempo" and not tempo:
            tempo = str(mido.tempo2bpm(msg.tempo))
        elif msg.type == "time_signature" and not time_sig:
            time_sig = f"{msg.numerator}/{msg.denominator}"
        elif msg.type == "key_signature" and not key_sig:
            key_sig = msg.key

    return (
        fix_encoding(title),
        fix_encoding(composer),
        fix_encoding(" | ".join(comments)),
        tempo,
        time_sig,
        key_sig,
    )


def extract_metadata(path):
    """Extract metadata from a single MIDI file."""
    try:
        mid = mido.MidiFile(path)
    except Exception as e:
        return {
            "filename": path.name,
            "title_raw": "",
            "composer_raw": "",
            "comments_raw": f"[ERROR READING FILE: {e}]",
            "tempo": "",
            "time_signature": "",
            "key_signature": "",
        }

    # Prefer Track 0, but fall back if needed
    track = mid.tracks[0] if mid.tracks else []

    title, composer, comments, tempo, time_sig, key_sig = extract_meta_from_track(track)

    return {
        "filename": path.name,
        "title_raw": title,
        "composer_raw": composer,
        "comments_raw": comments,
        "tempo": tempo,
        "time_signature": time_sig,
        "key_signature": key_sig,
    }


# ------------------------------------------------------------
# Write XLSX and CSV
# ------------------------------------------------------------

def write_xlsx(rows, path):
    wb = Workbook()
    ws = wb.active
    ws.append(list(rows[0].keys()))
    for row in rows:
        ws.append(list(row.values()))
    wb.save(path)


def write_csv(rows, path):
    with open(path, "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=rows[0].keys())
        writer.writeheader()
        writer.writerows(rows)


# ------------------------------------------------------------
# Main
# ------------------------------------------------------------

def main():
    midi_files = sorted(MIDI_FOLDER.glob("*.mid"))
    rows = []

    print(f"Scanning {len(midi_files)} MIDI files...")

    for path in midi_files:
        print(f"Extracting: {path.name}")
        rows.append(extract_metadata(path))

    if rows:
        write_xlsx(rows, OUTPUT_XLSX)
        write_csv(rows, OUTPUT_CSV)

    print("\nRaw metadata written to:")
    print(f"  {OUTPUT_XLSX}")
    print(f"  {OUTPUT_CSV}")
    print("\nExtraction complete.")


if __name__ == "__main__":
    main()

 

Project type
Scripting