Photo and voice input

Asking by voice, or reading an alarm code from a photo of the HMI, on the edge PC.

Two optional ways to fill the question box, for gloved hands and noisy floors. Both are processed on the edge PC or the plant network, and both only fill the question box: nothing is asked until you confirm the text.

Voice

Press Speak a question, speak, then Stop recording (or choose or a recording to upload an audio file). The recording is sent once to the site's own speech-to-text server on the plant network; the transcript appears in the question box for you to correct and send. The recording is not stored. Only a local or private server address is accepted, so a recording never leaves the site.

HMI photo

Choose HMI photo and take a picture of the alarm banner. The edge PC reads the text with OCR and shows Read from the photo: with the codes it found; codes that match an alarm definition come first. Press Use code to put it in the question. If no code was found, type it instead. The photo is read in memory and not stored.

Turning them on

An administrator turns each on in site.yaml:

inputs: {voice: "http://whisper:9000", photo: true}

The voice server is any OpenAI-compatible speech-to-text endpoint on the plant network (for example a local Whisper server). Photo reading uses Tesseract on the edge PC. When neither is set, the buttons are not shown.