Imagine climbing a tall mast, noticing a problem, and fumbling with a smartphone or a notepad to record the details.
Common issues range from woodpeckers pecking the masts or graffiti to more-serious problems like high-voltage electrical components failing.
Now imagine doing that in the middle of winter. (During warm weather, power companies can construct new parts of the electrical grid, so inspection work tends to get shunted to the colder months.)
Powel AS, a supplier of software solutions to the energy industry (among others), decided to take matters out of the hands of the field engineers—literally.
With a Xamarin.Forms app named André, engineers can use their voices to send inspection reports directly to the back-end database at the main office. All it takes is Google speech recognition, the Microsoft Bot Framework, and Language Understanding Intelligent Services (LUIS).
The workflow is simple:
“A field worker talks to André (the mobile app) in his natural language. The speech is transformed into text by the Google speech recognition engine and returned to the mobile app. The app sends the text for translation and receives fully translated text in return. The app then passes the translated text to LUIS, which captures the intent and entities and sends the response back to the mobile app. The mobile app sends the detected intent and entities (including the field worker ID) to the Bot Framework, which sends a request to the Powel back end. The appropriate response is returned to the mobile app, which translates the text into the field worker’s language and speaks the text aloud.”
OK, fine—it’s not simple, but using it is. And André uses the phone’s GPS to identify the facility and provide lists and information specific to that location.
Even more impressive: Powel and Microsoft developed the proof-of-concept bot in only three days. The code is available on GitHub at readyforchaos/Powel-power-station-inspection-bot, and a technical case study provides plenty more details: Using CaaP and André, the voice-driven assistant bot, to enhance on-site inspection.
What other roles could use this approach? Off the top of my head, I think of vehicle mechanics, firefighters, delivery drivers, emergency medical personnel—indeed, anywhere your hands are busy but you need information now.
This article was previously published on LinkedIn Pulse (https://www.linkedin.com/pulse/power-chatbots-dail-magee-jr)