Local web application

MUDIDI includes a localhost dashboard for production inference without YAML or CLI flags. The pipeline and files stay on the computer running MUDIDI; only model requests are sent to the selected provider.

Start the dashboard

Docker provides the same reproducible MUDIDI environment on macOS, Windows, and Linux. Install Docker Desktop on macOS or Windows, or Docker Engine with the Compose plugin on Linux. Start Docker, clone this repository, and run from its directory:

docker compose up --build

The first build downloads and installs the image dependencies. When the log prints MUDIDI dashboard: http://localhost:8000/, open that address. The Compose port is published only on 127.0.0.1; do not change it to a public interface because this is a local, single-user application.

For later starts, no rebuild is required:

docker compose up

To run in the background and follow its logs:

docker compose up -d
docker compose logs -f

Stop a foreground run with Ctrl+C. Stop and remove the container with:

docker compose down

The mudidi-data/ directory beside compose.yaml persists the database, encrypted credentials and encryption key, presets, managed uploads, worker artifacts, and generated outputs. Keep that directory private and back it up as one unit. Rebuilding or replacing the container does not delete it.

If Docker reports that it cannot connect to the Docker daemon, start Docker Desktop (macOS or Windows) or the Docker service (Linux), wait until the engine is ready, and repeat the Compose command.

Native dashboard with uv

Native runs use PyMuPDF for PDF page extraction; no separate PDF system tool is required.

Then install the web dependencies and start MUDIDI:

uv sync --frozen --extra web
uv run mudidi web

Request and upload sizes are unlimited by default. To cap them, set one or both limits and leave request headroom for multipart framing:

uv run mudidi web \
  --max-request-bytes 115343360 \
  --max-upload-bytes 104857600

--max-request-bytes applies to the complete raw HTTP request. When both are set, the request limit must be greater than --max-upload-bytes.

Save the API key for your model provider under API credentials on the New Run screen by clicking Save key beside that provider. MUDIDI opens http://localhost:8000. It binds to loopback and is not intended for public or LAN deployment. Use --no-browser or --port when needed.

MUDIDI New Run dashboard showing the Input step

The dashboard has a dark navigation sidebar (New run, Active runs, Run history, Saved presets), a numbered six-step tab bar across the top of New run, and a live Run summary rail on the right. The header shows a status chip that follows the selected billing mode (subscription accounts signed in, or provider keys available) and, once at least one preset is saved, the preset picker. Dark mode at the bottom of the sidebar switches between the light and dark themes. On narrow screens the sidebar moves to the top and the step tabs shrink to their numbers; the same form fields and validation rules apply in every layout.

Create a run

The New Run screen is a six-step wizard:

  1. Input — upload exactly one dictionary PDF, enter the dictionary pages, optional introduction pages, and the output directory.
  2. Pipeline — choose one of the three supported workflows, an existing output policy, and whether Stage 1 marks bold and italic text.
  3. Context — optionally describe the dictionary in the Dictionary Profile, supply MDF parsing-guide inputs, and, under the advanced toggle, add Stage 1 and Stage 2 instructions.
  4. Model — manage the local provider credential, select the provider, model, and reasoning settings, and configure batch size.
  5. Agentic — leave the agentic loop off, or turn it on and adjust its evaluator and correction settings.
  6. Review — submit the complete form for authoritative server validation, then inspect the server-rendered, non-secret review before starting the run.

The Pipeline step comes before Context so Context shows only the inputs the chosen pipeline uses; for example, the character inventory is hidden for Parse into MDF. The step buttons, Continue, and Back move between configuration panels without validating or clearing values. This allows the five configuration steps to be completed in any order. At final Review run, the browser performs whole-form constraint validation across all enabled fields before sending the complete multipart form to /runs/preview. If a field is invalid, the wizard opens its panel, displays the validation message, and focuses the first invalid control. Browser checks are only an early convenience: the server remains authoritative for required fields, PDF page bounds, profile completeness, model settings, and the rest of the production configuration. A rejected submission always returns to New Run with safe, user-facing validation details. Only errors rendered in wrappers marked for automatic routing open their panels; selecting that panel does not move keyboard focus. Provider errors and all other errors without automatic routing remain in the New Run summary.

The Input step asks for:

  1. one dictionary PDF;
  2. the required PDF dictionary pages to process. You may enter one page (5), one range (10-20), comma-separated pages (1,5,9), or a combination (1,5,10-20);
  3. an output directory on the same computer running MUDIDI;
  4. optional introduction pages.

Page numbers are 1-based: zero, negative numbers, descending ranges, and pages beyond the uploaded PDF's page count are rejected. Browser-selected files are copied into an input bundle owned by the run so review, restart, and resume do not depend on the original browser selection.

Each accepted upload is staged in a local server-side draft before the full form is validated. After a rejected form submission, the dashboard shows each retained filename; select another file to replace it, or use Remove to discard it. A standard browser still cannot refill a file picker, so the retained local copy is used when the form is resubmitted without choosing a replacement. The same-tab form state restores the draft identifier after a reload. Inactive drafts expire after one hour; successfully prepared runs take ownership of their selected uploads.

A standard browser cannot disclose an arbitrary absolute folder path to a localhost server, so the output directory is a text field. When MUDIDI runs natively with uv, Choose folder… beside the field opens the operating system's folder dialog on the same computer and fills in the selected path. The button is absent in Docker and on machines without a supported dialog; type the path instead.

The Pipeline step presents these mutually exclusive choices:

  • Complete digitization — transcribes the dictionary, infers and reviews an MDF parsing guide, then parses the transcription into MDF;
  • Transcription only — produces faithful flat Stage 1 text without MDF parsing;
  • Parse into MDF — uses an existing transcription, infers and reviews a dictionary-specific MDF (Multi-Dictionary Formatter) guide, and emits MDF records.

The existing output policy is separate: Resume compatible artifacts reuses compatible work, while Overwrite existing artifacts replaces it. The selected pipeline determines which later inputs and model controls are active. The dashboard always uses flat Stage 1 output.

Stage 1 typography (shown when the pipeline includes Stage 1) defaults to Mark bold and italic: the transcription wraps confidently bold text in <b>…</b> and italic text in <i>…</i>, which often signals entry structure such as bold headwords. These tags appear in Stage 1 transcription files and in the Page Viewer & Editor; MDF parsing removes them when it writes MDF lines. Choose Plain text to transcribe characters only. Changing this setting does not invalidate existing pages under Resume compatible artifacts; choose Overwrite existing artifacts to re-transcribe them.

OCR hints, column mode, and expert OCR/VLM backends remain available through YAML and the CLI but are intentionally absent from the dashboard.

MUDIDI dashboard Pipeline step

The final Review page is rendered by the server, not another client-side wizard panel. It groups the validated non-secret values under Input, Pipeline, Model, and Agentic, reports MDF parsing-guide review requirements, and provides the start action. Complete and MDF-parsing runs pause later for explicit MDF parsing-guide approval; that human checkpoint is distinct from this pre-run configuration review.

The grey values beginning with ex: are examples only; they are not submitted as values. Each one shows the accepted forms: a range, a list, or a mix, for example ex: 30-35 or 30, 32, 35 or 30, 33-35.

The web dashboard does not accept page images, multiple files, or a folder of images. Those input modes remain available through YAML and the CLI.

Dictionary Profile

The optional Dictionary Profile is the first section of the Context step and can improve extraction accuracy. It asks for:

  • headword language and script;
  • one or more paired translation, gloss, or definition languages and scripts;
  • a character inventory for each language-script pair, used by Stage 1 to constrain transcription to valid characters (hidden when the pipeline has no Stage 1);
  • a free-form description of the page arrangement;
  • the information types found in entries.

Every profile question is optional. Leave the whole section blank, or provide only the context you know. Language and script are paired: when a headword or target language is supplied, its script is required; target languages must also have matching scripts. Page layout, information types, and character inventory can be supplied independently. The profile is guidance, not source text, and MUDIDI still checks the scanned page. It does not strictly limit discovery to the information types you enter: the model may identify additional entry structures and rules visible in the dictionary.

Additional context

After the profile, the Context step shows the MDF parsing guide section for pipelines that run Stage 2. It offers two choices:

  • Create a guide for me (the default) — MUDIDI studies sample pages, proposes a guide, and pauses the run so you can review and edit it. The optional Pages to learn from field names the sample pages. Left blank, every page entered under Dictionary pages in step 1 is used (not the whole uploaded PDF). If that is hundreds of pages, enter two or three typical pages, because every sample page is sent in one request.
  • Use a guide I already have — upload the guide JSON from an earlier run of the same dictionary (mdf_parsing_guide.json in that run's output folder). The run uses it as is and skips the review step.

Only the field for the selected choice is shown and submitted.

Stage instructions sit behind the Advanced: stage instructions toggle at the bottom of the step. It starts off. Turning it on reveals:

  • Stage 1 instructions entered as text or uploaded as one .txt, .md, or .pdf file;
  • Stage 2 instructions entered as text or uploaded as one .txt, .md, or .pdf file.

Turning the toggle off only hides the section: entered instructions are kept and still submitted. While hidden, the toggle shows how many are set, and the run summary's Instructions row names the stages that have them. The toggle opens automatically when a loaded preset or restored session contains instructions, or when an instruction field fails validation.

Dictionary, introduction, and representative MDF parsing-guide page numbers must be positive Arabic numbers within the uploaded dictionary PDF. Representative MDF parsing-guide pages must also be included in the selected dictionary pages. Instruction PDF page numbers must exist within their own attachment. Roman numeral page specifications are not accepted.

For an instruction attachment, select Upload instruction file instead of Type instructions. TXT and Markdown files must contain non-blank UTF-8 text and are limited to 20,000 characters. PDF attachments retain document layout, images, and typography. An optional PDF page specification selects one page, an ascending range, comma-separated pages, or a combination; leaving it blank uses every page in the attachment.

Stage 2 instructions also have a scope: Pass 1 only, Pass 2 only, or Both passes. The default is both. Switching away from a selected file asks for confirmation before clearing it. Review pages and saved presets show the attachment name, type, selected PDF pages, and Stage 2 scope without exposing file contents.

Uploaded and typed instructions are copied into the run-owned input bundle and passed through the same prompt-guide execution path used by YAML and CLI runs.

MDF parsing guide and MDF manual

These names refer to different things:

  • MDF parsing guide is inferred by the LLM for this particular dictionary. It describes the MDF markers and structural rules that Stage 2 should use.
  • MDF manual is bundled reference text from the SIL Toolbox MDF Reference Manual.

The dashboard has no MDF manual setting. Every dashboard run whose pipeline includes Stage 2 Pass 1 adds the bundled manual text to the Pass 1 parsing-guide discovery system prompt. Pass 1 is a single request per run, so this adds approximately 30K input tokens once, not once per page. Pass 2 never receives the manual. The review page shows the manual as used or not used. To run Pass 1 without the manual, use YAML (input.mdf_manual: false, the default) or the CLI (--no-mdf-manual).

The repository owner has confirmed redistribution permission for the bundled text. No manual PDF upload is needed. The manual does not replace the dictionary-specific MDF parsing guide.

The existing parsing-guide file picker uses the same themed upload control as instruction attachments, with the selected filename displayed beside it.

Agentic loop

The agentic loop is the fifth wizard step. After each page is produced, an Evaluator model checks the output against its source. If it finds problems it proposes targeted edits, each naming a line, the exact current text, and the replacement. An Editor model then verifies every proposed edit against the source and approves or refuses it with a reason. Approved edits are applied as exact replacements, and the Editor's verdicts go back to the Evaluator, which checks the page again. This repeats until the page is accepted, the iteration cap is reached, or the loop detects that it is no longer making progress. For a page that cannot be repaired line by line, the Evaluator can instead request a full redo: Stage 1 re-transcribes the page from the image, and Stage 2 parses it again from the transcript. It defaults to Off because it adds model calls and cost. Select On to reveal the Custom loop settings panel. The applicable Stage 1 and Stage 2 checks start enabled; you can disable either one, then configure Maximum Iterations (default 3; one iteration is one Evaluator-to-Editor round), minimum retry confidence, and the Evaluator and Editor models and reasoning. Choosing different models for the two roles makes their checks more independent.

Only stages in the selected pipeline can be verified. Controls for inactive stages are disabled in the browser, and the server intersects submitted stage choices with the active pipeline rather than trusting a forged inactive value. The Stage 1 and Stage 2 production calls still use the one synchronized run-level provider described below; Evaluator and Editor settings retain their existing optional role-specific model controls.

Models and providers

The Model step begins with the selected provider's credential card, then shows model, reasoning, and batch-size controls for active stages. There is one authoritative run-level provider form value. The Stage 1 and Stage 2 provider selectors are synchronized presentations of that value, not independent per-stage providers: changing either selector updates the other, and the same single provider is submitted for the run. Existing multipart field names, including provider, stage1_model, stage2_pass1_model, stage2_pass2_model, and their reasoning fields, remain unchanged; the wizard changes visibility and presentation, not their meaning.

Stage 1 exposes its model and reasoning when transcription is active. Stage 2 uses one shared model and one shared reasoning selection by default. The shared values are written to both existing Pass 1 and Pass 2 controls, so both Stage 2 calls use the same choices. The summary identifies this as Shared model.

For separate Stage 2 choices, select Advanced · split passes. The action copies the current shared model and reasoning into both pass cards and replaces the shared presentation with:

  • Pass 1 — examines representative pages to infer the dictionary-specific MDF markers and entry structure. Its generated guide is reviewed before conversion;
  • Pass 2 — applies the approved Pass 1 guide to each authoritative Stage 1 transcription and emits the per-page MDF records. This is the high-volume conversion pass.

Each split-pass card has its own model and reasoning controls, while the run-level provider remains shared. The summary changes to Separate pass models and shows both models. Use one Stage 2 model returns to shared mode and reuses the previous shared selections. Independently cached split-pass values survive toggling only within the current page instance; because shared values replace both pass controls before session storage persists them, a reload or navigation restores those shared replacement values rather than the independent split-pass choices.

With API-key billing, MUDIDI loads the selected provider's current account model list on the server, caches a successful result for 15 minutes, and groups the choices into stage recommendations and additional account models whose compatibility is not verified. Refresh models bypasses that cache. A missing key or unavailable provider leaves the bundled fallback usable and reports the degraded state without exposing provider errors or credentials.

Subscription billing instead lists the models advertised to the authenticated OpenAI, Google, or Claude account. When the provider names its own default and recommended models (Google does), they appear first under Recommended by Google, default first, and the Model step preselects that default; every other advertised model follows under Other models from your subscription. Without such signals, all models share one group. Google then places Pro models before Flash models. Within each Google family, and for other providers, release timestamps take precedence; provider order and numeric model revisions provide deterministic fallbacks, so revision 3.10 sorts ahead of 3.9. Subscription mode has no Other model escape hatch: preview and run submission reject model identifiers absent from the current authenticated catalog.

A provider catalog can list models the provider no longer serves; Google's lists Gemini 2.5 Pro to accounts whose every request to it returns HTTP 503. Before the Review page, MUDIDI therefore sends each chosen subscription model one tiny request ("Reply with the single word: ok") with the selected reasoning level. A refused model is rejected with an error on its field; a model that answers is not tested again for ten minutes. If a stage's model field arrives blank, MUDIDI uses the first of up to three catalog models, starting with the provider's default, that answers. These test requests count toward the subscription's usage like any other request.

Google's subscription catalog includes internal effort-qualified and agent aliases. MUDIDI collapses -low, -medium, and -high variants into one logical model with only the available reasoning levels. Fixed (High) agent aliases expose only High. When Google declares several compatibility keys with the same available canonical tagDescription, MUDIDI shows that canonical model once. Distinct native IDs without a canonical tag remain selectable and include the native ID for disambiguation.

API-key and custom-provider modes retain Other model for manual LiteLLM identifiers. MUDIDI adds the direct-provider prefix when required. The optional OpenRouter Provider slug pins an endpoint preference, while blank uses automatic routing. Existing API-key selections that disappear from a later provider response remain available as an explicitly labeled current selection.

Reasoning choices are model-specific and use the portable order Minimal, Low, Medium, High, Xhigh, and Max. Known models expose only their reviewed or provider-advertised subset. Unknown manual/API-key models expose all six choices. Legacy none values resolve to the known model's lowest effort, or low for an unknown model, rather than disabling reasoning.

Complete-digitization workflow

For a complete run, the dashboard displays these steps in execution order:

  1. Stage 1 — Transcription processes all selected dictionary pages.
  2. MDF parsing guide discovery samples the configured representative pages from the Stage 1 transcriptions. If no pages were specified, MUDIDI uses every selected dictionary page.
  3. Review parsing guide pauses the run for human review and approval.
  4. Stage 2 — MDF conversion converts each Stage 1 transcription into MDF using the approved guide snapshot.

The Overview reports the active stage, completed-page count, current page, and failure details. Updates arrive while the run is active. Select Cancel run to interrupt an active worker. An interrupted run can be resumed from its safe checkpoint, and compatible completed artifacts are reused when the output policy is Resume.

MDF parsing guide review checkpoint

Complete and MDF-parsing runs pause when Stage 2 infers an MDF parsing guide. Open MDF parsing guide, review its markers and guide rules, make any edits, and select Approve and continue MDF parsing.

Saving a draft is not approval. Page parsing cannot start without a server-minted approval bound to the exact run, review version, immutable snapshot, digest, and approval time.

After approval, the dashboard page becomes read-only. It shows the immutable snapshot actually used by Stage 2; editing it after the run finishes cannot change the completed output or the guide used by a future run. To reuse revised rules, save or upload the intended parsing-guide JSON when configuring a new run.

A user-uploaded existing MDF parsing guide follows a different path: MUDIDI copies it into the run-owned input bundle, validates its JSON and marker format, and uses it directly without Pass 1 discovery or a human checkpoint. The guide is validated again when Stage 2 loads it. Uploading a guide therefore means the user is supplying the intended parsing rules, while malformed files still fail safely before MDF parsing.

Monitor, inspect, and correct pages

Every run page shows the same workspace tab bar, with the current view highlighted. A view that is not available yet, such as MDF parsing guide before a guide exists, appears greyed out instead of disappearing. Run views include:

  • Overview — durable status, progress, resume, and cancellation.
  • MDF parsing guide — structured review before approval and the read-only approved snapshot afterward.
  • Page Viewer & Editor — the rendered source page beside editable generated text, with previous/next controls and a slider across processed pages.
  • Live Logs — bounded diagnostics with known keys redacted, including PDF-splitting progress.
  • File Artifacts — downloads constrained to the validated output directory.
  • Usage — reported token and cost totals.

The Page Viewer & Editor becomes useful before the whole pipeline finishes:

  • when only Stage 1 exists for a page, it shows the source and Stage 1 transcription;
  • when Stage 2 finishes that page, it shows both Stage 1 and Stage 2 MDF;
  • newly completed pages become available automatically while a run is active;
  • only processed pages appear in the slider.

Saving changes replaces the corresponding Stage 1 and/or Stage 2 text file in the run's configured output directory. A Stage 1 correction does not regenerate an existing Stage 2 file, so correct Stage 2 separately or rerun MDF conversion when consistency matters. Avoid editing the page currently being written by the worker because its output may replace your change.

Run history

Run history can be searched by run ID and filtered by status or provider. Each inactive run has a Remove action directly in the list, and Delete all history removes all inactive dashboard records and their managed inputs. These actions do not delete generated files from the configured output directory. Active runs must first finish or be cancelled.

Runs and events survive restart. A run active when the app stopped becomes interrupted and must be resumed explicitly. Presets own independent copies of their managed inputs, so deleting or cleaning a source run does not break a saved preset.

Credentials and local data

On the Model step, Subscription billing appears above API-key billing and is selected by default for a new run. API-key billing displays only the selected run-level provider's credential card, with its saved status, masked input, reveal button, and Save key. Selecting another model provider swaps the visible card without changing or deleting any stored credentials. Subscription billing hides the API credential fieldset and displays only the selected subscription provider's account card; switching back restores the matching API card.

Google subscription login uses Google Antigravity's installed-app OAuth registration by default, with the same five scopes and loopback callback used by Antigravity clients. No OAuth setup is required for normal dashboard login. A deployment with its own compatible Antigravity registration may set:

MUDIDI_GOOGLE_OAUTH_CLIENT_ID=your-client-id.apps.googleusercontent.com
MUDIDI_GOOGLE_OAUTH_CLIENT_SECRET=your-client-secret
# GOOGLE_CLOUD_PROJECT=your-project-id

The client ID and secret are optional overrides. GOOGLE_CLOUD_PROJECT remains optional for accounts that require an explicit Cloud Code Assist project. Keep override values in .env or the parent process environment, not in run YAML. Subscription workers do not inherit an override client secret.

The API credentials section accepts Gemini, OpenAI, Anthropic, and OpenRouter keys. Click Save key to persist an entered value. Inputs are masked by default and the eye button explicitly reveals a saved value. The existing credential contract is unchanged: the key is sent as the api_key field to POST /credentials/{provider}, a saved key is revealed only after the same-origin POST /credentials/{provider}/reveal action, and removal uses POST /credentials/{provider}/delete. These actions do not put keys into run form fields, presets, resolved configuration, logs, command lines, or URLs.

Provider keys are encrypted before their ciphertext is written to SQLite. MUDIDI stores the encryption key separately at .credential-key in the same private data directory. This protects a copied database from exposing plaintext credentials, but anyone who can read both files as your local user can decrypt them. Keep the complete directory private. The dashboard does not fall back to .env; .env remains the credential mechanism for CLI and YAML workflows. The encrypted storage, field names, endpoint paths, and reveal semantics are unchanged by the wizard presentation.

When MUDIDI uses LiteLLM directly, there is no separate LiteLLM API key. The model identifier selects a provider and LiteLLM uses that provider's key—for example, an OpenAI model uses the saved OpenAI key. A LiteLLM virtual or master key is relevant only when connecting to a separately hosted LiteLLM Proxy.

The browser uses the existing session-storage behavior for wizard continuity: non-file, non-password form values persist only in the current browser tab. File inputs and password/key values are intentionally excluded, so session storage does not restore them and does not carry them across tabs or browser restarts. Wizard presentation state, including the shared/split Stage 2 choice, is kept separately from the form values.

Web data defaults to:

~/.local/share/mudidi/
├── mudidi-web.sqlite3   # run history, presets, encrypted key ciphertext
├── .credential-key     # local encryption key; keep private
├── presets/             # preset-owned managed inputs
└── runs/                # run-owned inputs and worker artifacts

Override the complete data directory with:

uv run mudidi web --data-dir path/to/private-app-data

Generated dictionary files remain in the selected output directory. In Docker, an output such as outputs/web-output is written to the repository's host directory at outputs/web-output; Compose mounts ./outputs at /app/outputs. Paths under /data are persisted in the host's mudidi-data/ directory. Other absolute host paths are rejected because Docker cannot access an unmounted host directory. With uv, output paths refer directly to the host filesystem.

Running several runs at once

You can start a run while others are working; there is no limit on how many run at once. Active runs lists every run in progress with its stage and progress; select a run to open it. A working run has a Cancel action. A run that is waiting for you to review its parsing guide stays in the list with a Review guide action.

  • A run is refused when another in-progress run (working, or paused for parsing-guide review) writes to the same output folder. The Review page says which run holds the folder. Wait for that run, cancel it, or prepare the new run with a different output folder.
  • Runs that use the same provider login share that login's rate limit. The Review page and the run page warn about this. Such runs still start, but they may each go slower than they would alone.

Troubleshooting

  • API credential required — save the matching key under API credentials on New Run.
  • Another inference worker is active — finish or cancel the current worker.
  • Awaiting MDF Parsing Guide Review — review and explicitly approve the guide; this pause is intentional.
  • Interrupted — inspect the run and explicitly resume it.
  • Request body too large or uploaded input is too large — the dashboard was started with --max-request-bytes or --max-upload-bytes. Restart it with larger values or without those flags; sizes are unlimited by default.
  • Address already in use on 127.0.0.1:8000 — another dashboard process is already listening. On macOS or Linux, inspect it with lsof -nP -iTCP:8000 -sTCP:LISTEN, stop the listed process with kill PID, or start MUDIDI on another port with uv run mudidi web --port 8080.
  • Docker cannot connect to the daemon — start Docker Desktop or the Docker service and wait for docker info to succeed.

For advanced options omitted from the dashboard, use the YAML configuration guide and CLI reference.