Voice Transcriptions

The Voice Transcriptions module turns mobile diary dictation into a field capture station. A supervisor dictates short diary notes hands-free — from the home launchpad or from inside a diary — and each recording is saved fire-and-forget, transcribed by OpenAI Whisper, and structured by GPT in the background. Notes accrue against the diary as a queue of pending changes; the supervisor (or an office reviewer) later reviews each proposed change field-by-field and applies it. Review and apply work symmetrically on the phone and on web admin.

This is the first AI-assisted data-entry module in Lacroo Pro and is the foundation layer for future dictation targets (prestarts, dockets, comments).

Overview

Who it’s for. Construction supervisors working a shift who would rather talk than type. The design goal is that a gloved supervisor can tap once, speak, and move on — many times across a shift — and reconcile everything later.

The mental model. Dictation is a capture station, not a form. Recording a note never blocks on review and never immediately writes to the diary. A note is captured, processed in the background, and lands in the diary’s Notes queue marked “Ready for review”. A supervisor can fire ten notes across a shift and clear the queue at the end — on the phone at the site, or handed off to the office to clear on web.

What it fills. Only diary-level fields — and only the free-text / notes / meta side of the diary. Activities, the worker roster, cost codes, and quantities are explicitly out of scope; those flow through dockets into the Award Engine and payroll, so human review is required. See Supported fields.

What it doesn’t do. No prestart dictation, no docket dictation, no comment dictation, no AI edits to activities or cost-coded items, no hand-editing of the transcript. Capture-while-fully-offline (beyond the failed-upload outbox) is out of scope. See Known limitations.

How it works (for users)

The capture flow (mobile)

1. Start a note. Tap Dictate diary note — the featured top option on the home launchpad — or Dictate on the bottom action bar inside an open diary.
2. Resolve the target diary. When dictating from home, the app resolves which diary the note belongs to (see Target-diary resolution). Inside a diary, the target is that diary.
3. Permission pre-prompt (first run). The first time you dictate, a short sheet explains microphone use before the OS permission dialog appears (matching Lacroo’s Apple-approved permission pattern). If you deny, the app offers a Settings deep-link.
4. Record. A record sheet shows the target diary, a pulsing mic, a timer (m:ss / 5:00), and a waveform. Speak naturally (“Site was Block 4 level 2, handover for nightshift is the scissor lift needs a tag, prestart ran 30 minutes…”). Recording continues even if the app is backgrounded — see Background recording.
5. Stop & save. Tap Stop & save. The audio uploads and the note is saved immediately — nothing is written to the diary yet. You are dropped onto a post-save branch: Record another (the default), Review notes, or Done.
6. Background processing. In the background the note moves through transcribing → extracting → completed. The diary’s Notes queue live-updates via WebSocket — no polling, no refresh.
7. Review later. Open the diary’s Notes queue (from the Voice-notes card on the diary, or the pending badge on the home diaries row). Each note shows its time, duration, a status chip, a transcript snippet, and “N suggested changes”.
8. Apply per-field. Open a note to see a per-field diff: the current diary value on the left, the AI-proposed value on the right, with an Accept / Reject toggle per field. You can reroute a value to a different field, and see any cross-reference chips. Tap Apply & next to write the accepted changes and advance to the next note, or Later to leave the queue untouched. Applying is confirmed explicitly on multi-note runs — there is no silent “accept all”.
9. Done. Applied notes are marked Applied (with a lightweight Undo); remaining notes stay Ready so the office can finish the queue on web.

Tip. The extraction snapshots the diary’s current values at record-time, so the AI proposes deltas — new information — rather than rewriting fields you already filled. Narrative fields (handover notes, activity description) are appended to; single-value fields (site location, prestart duration) are replaced.

Review on web admin

Every capability above except recording is available in the web admin diary detail. A pending-notes inbox lists the diary’s notes that are Ready for review; opening one shows the same per-field diff drawer with Apply & next. Web and phone apply through the same server endpoint, so merge semantics are identical and a note applied on one surface immediately reflects on the other.

Multiple notes per day

A supervisor is expected to record many notes across a shift against a single day’s diary. Notes accrue independently — each is transcribed and extracted on its own — and the queue is the single canonical place they collect. There is no limit on the number of notes per diary; each is a small, cheap capture.

Target-diary resolution

When you dictate from the home launchpad, the app decides which diary the note attaches to. Dates and shifts are evaluated in project-local time (never re-derived from UTC), avoiding cross-midnight errors:

• Exactly one draft diary for you as supervisor today → that diary is targeted.
• No diary yet and you hold a supervisor role → the app offers to create today’s draft (confirming the date and shift). It never silently creates one.
• Multiple diaries (day + night, multiple crews) or you are an admin/engineer who is not the diary’s supervisor → a small “which diary?” chooser appears. An admin is never silently made the supervisor_id of a fresh diary.

Dictating from inside a diary skips resolution — the note attaches to that diary.

Status lifecycle

A note walks a single status column from capture to a terminal state:

uploading → queued → transcribing → extracting → completed → { applied | discarded }
                                                     ↓
                              (any stage) → failed(stage) → (retry) → queued

applied and discarded are terminal: once a note is applied or dismissed, no late or duplicate background activity can flip it back. Dismissing a note is a soft operation (status = 'discarded') — the row and its audio are retained for audit, not hard-deleted. completed notes are what the pending queue and the home badge count.

Applying a note (merge model)

Applying is server-side — the server is the single merge owner, so phone and web behave identically. When you accept a set of fields and apply:

1. Apply-once. The note is atomically moved completed → applied (compare-and-set). If it was already applied — for example the office cleared it a second earlier — the apply is an idempotent no-op; the diary is untouched.
2. Draft guard. The diary must be in draft. If it was submitted between opening the note and applying, apply is blocked with a clear “revert to draft to apply” message.
3. Per-field merge, computed against the freshest diary state:
   • Narrative fields (activity_description, shift_handover_notes, post_publish_change_notes, textarea customs) → the proposed delta is appended to the current value.
   • Value fields (site_location, prestart_duration, numeric/date/dropdown customs) → the value replaces the current one (last-applied-wins).
4. References persisted. Accepted cross-entity references (issues, alerts, prestart, prior diaries) are written to the diary’s audit trail (diaries.metadata.voice_references).

This per-field strategy makes the multi-note flow deterministic: two narrative notes on an empty diary both land (no duplication), two value notes resolve last-wins, and re-applying the same note appends nothing a second time.

Background recording

The mobile app ships a native background-audio capability (UIBackgroundModes: ['audio'] on iOS plus staysActiveInBackground), so a recording keeps running if the app is backgrounded mid-note. If a call or Siri interrupts the recording, the app stops cleanly and saves what it captured. Because this is a native entitlement, background recording only works from the app binary that carries it — it cannot be delivered as a JavaScript-only over-the-air update.

If an upload fails (no signal, app killed), the audio and its target are kept in an on-device failed-upload outbox, and the app sends it again on its own:

• When. On the next app open, whenever the app comes back to the foreground, when the connection comes back, and right after another note saves. The Retry upload button on the record screen still works straight away.
• How often. It waits 30 seconds after a failure, then doubles the wait each time, up to 30 minutes between tries. There is no limit on the number of tries.
• Never twice. Every resend carries the note’s original idempotency key, so a note whose first upload did land is never added a second time.
• Whose phone. A queued note is only sent while the person who recorded it is signed in.
• When it stops. If the server refuses the note (for example, the diary was submitted in the meantime), the recording is no longer on the phone, or the note has waited seven days, the app drops it and says so: “A voice note couldn’t be uploaded. Please record it again.” When a queued note does go through, the app shows “Voice note uploaded.”

Supported fields

GPT is prompted with the exact list of fields below, each annotated with its current value. The model returns null for any field it doesn’t want to change, which the review UI treats as a no-op row (hidden from the diff). Narrative fields emit only the new information (a delta); value fields emit the value.

Core diary fields

Custom fields

Any diary-level custom field (i.e. entity_type = 'prestart_diary' in custom_field_definitions) of the following types is included in the extraction prompt:

• text (replace)
• number (replace)
• date (replace)
• dropdown (replace; value must match one of the defined options)
• multiselect (replace; value must be a subset of the defined options)
• textarea (append)

Custom fields of type checkbox, radio, time, or file are currently not extracted — they’re either easy to mis-interpret from speech or don’t make sense to dictate (files in particular).

Fields explicitly not filled

• Activities — the per-row activity list that feeds cost codes and eventually the Award Engine and payroll. Dictating a paragraph into the diary’s activity_description is supported; per-activity rows are not.
• Dockets (labour, plant, subcontractor, associated)
• Worker sign-ons
• Cost codes, quantities, line items
• Photos, file attachments, weather
• Any prestart field (the sibling entity)

Cross-reference context

Every dictation feeds the AI a full project context pack drawn from your live records — no configuration required.

What the AI sees on every extraction:

• Shift header — project, supervisor, diary date, shift type.
• Prestart (if the diary is linked) — planned activities, declared hazards, crew sign-on list, cost codes for this shift.
• Open issues & variations — up to 30 most-recently-updated open items with display IDs, type, status, the 5-stage timeline, financial impact, cause/impact/location, and the two most recent status changes.
• Active project alerts — up to 10 non-archived project alerts, plus any org-scoped mandatory bulletins.
• Recent diaries by the same supervisor — the last 3 non-draft diaries on this project, including every populated custom field value.

Cross-reference chips. When the AI quotes a specific issue, alert, prestart, or prior diary in a proposed value, the review UI shows a non-interactive chip row below that field: 🔗 IV-2026-0012 with a short caption explaining the link. On Apply, the accepted-field references are persisted to diaries.metadata.voice_references as an audit trail, and surface on the diary detail as a Voice dictation references panel.

Safety. References to entities the AI didn’t actually see in the pack are silently dropped server-side, so a fabricated ID never reaches the review UI. Dropped-count and emitted-count are both logged.

Permissions & availability

There is no per-org on/off flag. Voice dictation is available whenever the platform is provisioned with an OpenAI key; it is gated per-use on role + diary status + platform availability:

• Role. The caller must hold an authoring role on the project (supervisor and above). Clients and suppliers never see a Dictate entry point.
• Draft only. Dictation and apply require the target diary to be in draft status. Submitted, reviewed, and confirmed diaries are deliberately not dictatable — revert to draft to make corrections.
• Scope. Every voice operation runs under RBAC-v2 scope for diaries and re-loads the diary through the scoped path, so a scoped engineer only ever touches diaries they can actually see.

Tenant & project isolation. Every transcription query is filtered by org, project, and target diary, and the URL identifiers are cross-checked — a same-org user cannot read another project’s transcript or proposed values by ID.

Limits

• 10 MB per recording, enforced on the device and again in the Go service layer.
• 5-minute hard cap — the recorder auto-stops at five minutes. Longer narration is split across multiple notes (which is the intended capture-station pattern anyway).
• Too-short / empty audio is rejected before upload with a clear message.

Architecture (for developers)

Contract

All transcription operations stay nested under the diary resource, so tenant / role / membership checks come from the existing diary middleware. Every operation is scope- and IDOR-guarded (org + project + diary cross-checks).

The list status filter accepts pending | applied | failed | dismissed | all and hides discarded notes by default. resolve-draft takes { diary_date, shift_type } in project-local time; the supervisor is always the session identity (never the request body). The full contract lives in /openapi.yaml under the VoiceTranscription* schemas.

Data model

A single voice_transcriptions table holds the full lifecycle. Key columns:

• target_entity_type + target_entity_id — polymorphic target ('diary' today; prestart / docket / comment reserved for future phases).
• status — enum uploading / queued / transcribing / extracting / completed / failed / applied / discarded. Terminal states are protected by status guards on every writer, so no late activity can clobber a terminal row.
• audio_r2_key — pointer into R2 (see Storage).
• current_values_snapshot — JSONB of the diary values at record-time; feeds the extraction prompt so the model proposes deltas.
• raw_transcript, detected_language, duration_seconds — Whisper output.
• proposed_values — GPT output; JSONB with core and custom_field_values. Narrative fields carry deltas; value fields carry the value; unchanged fields are null.
• applied_at, applied_by, applied_fields, applied_surface — apply audit (applied_by is the reviewer, who may differ from the recorder; applied_surface is mobile or web). The surface comes from the X-Client-Surface header: the web Server Action sends web, and the mobile BFF route stamps mobile on everything it forwards. These columns back the Undo affordance.
• error_stage + error_message — populated only on genuine pipeline failures.
• temporal_workflow_id + temporal_run_id — operations debugging / audit.
• idempotency_key — unique per org; bound to the recording result so a re-tapped Stop cannot create two rows and reusing a key across different audio cannot return the wrong note.

A partial unique index (project_id, supervisor_id, diary_date, shift_type) WHERE deleted_at IS NULL enforces one live diary per slot and backs the resolve-draft get-or-create.

Storage

Audio uploads land in Cloudflare R2 at:

voice/{org_id}/{project_id}/{diary_id}/{transcription_id}/raw.{ext}

The key is persistent — audio is never deleted on the success path (needed for retries and future re-processing) and is retained even when a note is dismissed (the row flips to discarded; the audio stays for audit). The 10 MB cap is enforced in the HTTP layer and again in the service layer.

Temporal workflow

Upload commits → queued → the VoiceTranscription workflow starts with deterministic ID voice-{transcription_id} (a retry can’t spawn a duplicate). The pipeline downloads the audio, transcribes with Whisper, fetches diary context, extracts fields with GPT, and marks the note ready.

Two correctness properties matter:

• Error classification. Transient OpenAI errors (5xx / timeout) are retryable; permanent errors (bad audio, invalid tool args) are wrapped as non-retryable so they fail fast to failed(stage) instead of burning retries.
• Activity idempotency. A retry or worker restart resumes rather than re-billing OpenAI: transcription is skipped if raw_transcript is already set, extraction is skipped if proposed_values is already set.

An orphan sweeper (Temporal cron) marks any uploading / queued row that has sat in that state for 30 minutes as failed(persist), so a crash between INSERT and workflow-start can’t leave a permanent in-flight zombie inflating the pending count. The 30 minutes count from when the row was last (re)queued (updated_at), not from when the note was first recorded — so a two-day-old note sent back by Retry or Update with latest gets the same 30 minutes as a new one.

Real-time updates

The voice_transcriptions table fires pg_notify('project_changes', …) on INSERT and on status change; the Go WebSocket hub fans it to the org room. Clients invalidate the note, the diary’s notes queue (["voice-notes", diaryId]), and the pending-count badge (["voice-pending-count", projectId]), plus focus-refetch and pull-to-refresh, so queues and badges stay current after capture, apply, and dismiss. Apply also emits the diary NOTIFY so the diary form itself refreshes.

Extraction prompt

The GPT system prompt is assembled per-diary and includes the role definition, the list of core + custom fields with type hints and current values, a JSON schema enforced via function calling, and a rule set: preserve quantities and subcontractor names verbatim; emit only new information (a delta) for narrative fields; default to null when unsure; only change prestart_duration if explicitly dictated. The response is validated against the schema before being written.

Data and privacy notes

• Audio retention. Raw audio is permanently retained in R2 under a deterministic key — required for retry and re-processing. Dismissed notes also keep the R2 object (row → discarded, audio retained for audit).
• Transcript retention. raw_transcript and proposed_values are stored in Postgres, scoped to the org via the standard tenant-isolation pattern.
• Processor. OpenAI is the current processor for both Whisper and GPT. Audio and transcripts transit US regions. Australian data residency (Azure OpenAI) is on the roadmap but not shipped — do not represent voice-to-diary as data-resident.
• Logs. Per the Zero Trust Mandate, request/response bodies are PII-redacted before Axiom / Sentry. Transcription IDs, org IDs, project IDs, and stage names are safe to log; raw transcripts, audio content, and apply selections/reroutes are not logged.
• Tenant isolation. Every query on voice_transcriptions is filtered by org, project, and target diary; the R2 key path prefixes org_id so even a compromised key pattern cannot cross-tenant.

Troubleshooting

No Dictate entry point appears.

• Is the deployment provisioned with an OpenAI key? Voice dictation is unavailable on a keyless deployment (there is no per-org toggle).
• Is the diary in draft status? Submitted, reviewed, and confirmed diaries are deliberately not dictatable — revert to draft.
• Does the caller have an authoring role on the project, and is the diary in scope? Clients, suppliers, and out-of-scope roles don’t see it.

“Microphone permission denied.”

• iOS (Expo app): Settings → Lacroo → Microphone. The in-app deny path offers a Settings deep-link.
• Android: long-press the app icon → App info → Permissions → Microphone.

A note is stuck “Transcribing”.

• Background processing is asynchronous; give it a few seconds. If it never completes, the orphan sweeper eventually marks it failed, and you can Retry against the same audio.

The transcript is garbled or the extraction is nonsense.

• Loud environments (heavy machinery, wind) can defeat Whisper. Move somewhere quieter and re-record as a new note.
• Use Retry on a failed note to re-run the pipeline against the same audio. If it still produces garbage, dismiss the note and type manually.

“Revert to draft to apply.”

• The diary was submitted between opening the note and applying. Revert the diary to draft, then apply the queued note.

“Already applied.”

• The note was applied on another surface (phone/web) a moment earlier. This is expected and safe — apply is idempotent; the diary was written exactly once.

Cost

Voice dictation incurs OpenAI usage costs per note. Rough per-note figures:

• Whisper (transcription): USD $0.006 per minute of audio, billed in six-second increments.
• GPT (extraction): a typical diary extraction is well under 2,000 tokens total.

A 5-minute note therefore costs on the order of USD $0.05 all-in. Because the capture-station pattern encourages many short notes, per-shift cost scales with note count. These are pass-through to the OpenAI account configured on the Go API (OPENAI_API_KEY); Lacroo does not currently meter or bill this back per-org. Activity idempotency ensures retries and worker restarts do not double-bill.

Known limitations

• Diary targets only. No prestart / docket / comment dictation. The same pipeline would support them — future phases.
• No AI fill for activities. The per-row activity list, cost codes, dockets, and quantities are not extracted. Intentional — these feed the Award Engine and payroll, and a wrong quantity is far more expensive than a re-typed handover note.
• Limited custom-field types. Only text, number, date, dropdown, multiselect, and textarea custom fields are extracted. checkbox, radio, time, and file are skipped.
• No transcript hand-editing. You accept, reject, or reroute the AI’s proposal per field; you don’t edit the transcript text directly.
• Failed-upload outbox, not full offline capture. A failed upload is retained and retried on-device, but authoring while fully offline (beyond that outbox) is out of scope for this slice.
• Background recording needs the native binary. The UIBackgroundModes: ['audio'] entitlement only exists in the app binary that carries it — it can’t be delivered over-the-air. Users on an older binary won’t get reliable background recording.
• 10 MB / 5-minute caps. Longer narration is split across notes.
• US data region for OpenAI. Australian data residency requires Azure OpenAI (a later phase).
• No per-org usage metering. See Cost.

Key files