How to Test Voice Recognition in 4 Steps With Perfecto

Software Quality

Many industries have recently started incorporating some form of voice assistant or voice recognition system as part of their mobile apps. Banking is one major industry that has made strides in this area, adopting virtual assistants to help customers save time on basic banking tasks.

Retailers and grocery chains are hopping on the voice recognition bandwagon as well, incorporating major platforms such as Siri, Alexa, and Google Voice as another way to interact with customers.

While voice assistants bring opportunities to developers, they pose new challenges for QA teams as they do their mobile testing. This blog post will give a detailed overview of Perfecto’s support of test cases for voice recognition systems.

4 Steps to Testing Voice Recognition

Automating voice assistant flows as part of your end to end testing can be achieved, in part, with an open-source automation framework such as Appium.

There are four general steps to running automated tests for voice recognition software:
Screen navigation to voice assistant (supported by Appium).
Activate voice assistant (supported by Appium).
Say a voice command (requires advanced automation).
Validate screen/response (requires advanced automation).

While automated tests can handle the first two steps, the last two are challenging to perform as part of an automated script. Traditional libraries such as Appium do not support audio out of the box.

Fully Automated Voice Recognition Scenarios With Perfecto

For teams that wish to completely automate their voice recognition scenarios, Perfecto offers additional APIs:

Inject audio file. This allows testers to inject any audio file on a mobile device.
Audio speech to text. This translates the audio output from a device back into text for validation.

These additional audio capabilities enable teams to automate end to end test cases that include voice recognition with ease.

Example: Testing Google Voice With Perfecto

In the following example, you’ll see how you can automate a simple end to end flow that includes Google Voice with Perfecto.

There are three basic steps.

Open a device.
Click on Google Voice.
Inject a pre-recorded audio file.

For this sample script, we will be writing our code in Java and testNG.

First, you open a device in the Perfecto lab. This example shows a user selecting a Samsung Galaxy S8.

Next, use Perfecto’s “Object Spy,” which shows the application’s object properties, to access all the element locators.

Once you are in Object Spy, you can locate all elements on the screen, including system level objects like the Google Voice icon.

Then you can incorporate these locators as part of your automation script. Here is a sample snippet:

driver.context("NATIVE_APP");
			
			WebDriverWait wait = new WebDriverWait(driver, 30);
			AndroidElement voiceIcon = (AndroidElement) wait.until(ExpectedConditions.elementToBeClickable(driver.findElement(By.xpath("//*[@resource-id=\"com.google.android.googlequicksearchbox:id/search_widget_voice_btn\"]"))));
			voiceIcon.click();
			AndroidElement searchSong = (AndroidElement) wait.until(ExpectedConditions.elementToBeClickable(driver.findElement(By.xpath("//*[@resource-id=\"com.google.android.googlequicksearchbox:id/display_text\"]"))));
			
			injectAudio(driver, "PRIVATE:example_text_1_plus_1.mp3");
Here’s the function responsible for the audio injection
	private static void injectAudio(AndroidDriver<AndroidElement> driver, String path) {
		Map<String, Object> params = new HashMap<>();
		 params.put("key", path);
		 driver.executeScript("mobile:audio:inject", params); 
	}

Note that you can also upload your simple audio file to the Perfecto repository.

You are now ready to execute your automated test in testNG and view your results in the Perfecto dashboard.

When breaking down the test steps you can see exactly where the audio was injected and that the step passed successfully.

Bottom Line

Executing automated tests for voice recognition systems no longer needs to be a daunting task for QA. With Perfecto, teams can expand their test automation horizons with these additional audio capabilities at their fingertips.

In addition to voice recognition system testing, Perfecto supports a wide range of advanced mobile testing capabilities.

Watch Mobile Testing in Action

See how Perfecto works across mobile, web, and desktop testing, scriptless automation, AI-driven execution, and analytics in these guided demos.

See Perfecto in Action