How to Develop a Label Detection App with Rust and Amazon Rekognition

Using the new, generally available AWS SDK for Rust, you can interact with the Amazon Rekognition service and perform label detection. Label detection will analyze an input image file and provide a series of descriptors (labels) that classify the image into various categories. You can use this data to group images together in a library, for security monitoring purposes, and many other use cases. 

In this blog, we'll take a look at how you can use the Rust programming language to build a high-performance CLI tool that runs label detection, using the AWS SDK for Rust. We'll be building the app in the GitHub Codespaces service, to minimize the amount of environmental setup required.

Understanding Label Detection with Amazon Rekognition

Object detection is a specific computer vision task, within the larger machine learning space. An object detection model is typically pre-trained on a data set, and attempts to search for those objects as patterns in a raster image file. Label detection is similar to object detection, however the resulting descriptors (labels) apply to the entire image, not necessarily a specific sub-region of the image.

There are manifold use cases for label detection. A couple of those include manufacturing quality control or threat detection with security cameras.

Amazon Web Services (AWS) offers a managed service that enables you to perform object detection with ease. This managed service is known as Amazon Rekognition. You can use any of the Software Development Kits (SDK) for AWS to invoke the API in Amazon Rekognition that performs object detection, including the Rust SDK for AWS.

Let’s explore how you can build a simple Rust application that feeds an image file as input into Amazon Rekognition, runs label & object detection, and prints out the label results.

1. Set Up Your Rust Project

The first thing we need to do is create a new Rust project. I usually place new projects under a $HOME/git directory.

mkdir ~/git

cargo new ~/git/aws-rek-test

You’ll also need to install a couple of crates for the AWS SDK for Rust.

The generally available (GA) version of the AWS SDK for Rust was released in Q4 2023, and requires that you specify an API “behavior version.” The easiest way to accommodate this requirement is to install the AWS crates with the behavior-version-latest feature enabled. Otherwise, you will have to specify the behavior version in your Rust code.

cd ~/git/aws-rek

cargo add aws-config --features=behavior-version-latest

cargo add aws-sdk-rekognition --features=behavior-version-latest

You’ll also need the tokio async runtime, which is commonly used for async projects. Since the AWS SDK is all based on Rust async, you’ll need to install it.

cargo add tokio --features=full

You can open the project directory in Microsoft Visual Studio Code, for an improved editing experience.

2. Configure AWS Credentials

In your Rust application, we need to set the credentials that will be used for AWS account access. AWS SDKs for various languages support a variety of mechanisms to expose credentials to it, including environment variables.

However, another common mechanism is to store credentials in a specially crafted AWS “credentials file,” where the SDK can load the values from. This helps avoid accidentally committing credentials into your application source code.

The default path that AWS SDKs search for the credentials file is under $HOME/.aws/credentials. This file contains an INI file format, and can contain multiple, named “profiles” to connect to different AWS accounts, using separate sets of credentials. If you don’t want to specify a profile name, the AWS SDK automatically looks for a profile named “default.”

Let’s create the AWS directory and then edit the credentials file.

mkdir ~/.aws/

vim ~/.aws/credentials

Use this template to specify your AWS credentials in that file.

[default]

aws_access_key_id=<paste_yours_here>

aws_secret_access_key=<paste_yours_here>

region=us-west-2

NOTE: Due to a known inconsistency between the AWS CLI and the AWS SDKs, the INI file property (key) names must be specified using lowercase letters. The AWS CLI will work with uppercase property names, but the Rust SDK will not.

3. Update Main.rs File

Update your main.rs file to look like the following. Note that we are using the tokio::main macro to act as an async executor, and using the async keyword to make the main function asynchronous.

#[tokio::main]

async fn main() {

}

This is a common pattern that you’ll see in any async Rust applications, including those developed with the AWS SDK.

4. Build Rekognition Rust Client

With the Rust SDK for AWS, you typically build a client using the crate for the specific service you’re working with, in this case, Rekognition.

Add the following code to the body of your main function.

use aws_sdk_rekognition as rek;

let cfg = aws_config::from_env().load().await;

let rek_client = rek::client::Client::new(&cfg);

The first line just creates an alias to the aws_sdk_rekognition crate, so we can access it via rek instead. The second line loads the default AWS SDK configuration, which loads the credentials and region from the default ~/.aws/credentials file path. The third line creates the Rekognition service client object.

Now that you’ve constructed the AWS configuration and Rekognition client, you can invoke various API calls against the Rekognition service.

5. Detect Image Classification Labels

One of the computer vision tasks we can perform with Rekognition is to get an array of “labels” that describe the image. This metadata could be useful in building an image search service, such as Google Photos, that allows users to find photos based on descriptors. For example, searching for “car” should return photos containing common vehicles, or searching for “plants” should return photos of trees, shrubs, flowers, and other types of plants.

The Rekognition client in Rust provides a detect_labels() function, which corresponds to the underlying DetectLabels REST API call. If you explore the documentation for this REST API call, you’ll notice that it requires you to pass in an image payload, or a pointer to an image file in Amazon S3 storage.

Let’s specify a local file for now, to avoid adding a dependency on another service. We’ll start off by loading that file into memory.

let input_file = std::fs::read("test_file01.jpg").unwrap();

Now we have a variable named input_file that points to a Vec<u8>. Note that if the file doesn’t exist, your Rust program will panic. We won’t worry about implementing robust error handling for now.

Next, we need to wrap the file in an AWS Rekognition construct called a “blob.” This type is exported from the module named primitives, under the Rekognition crate.

let input_blob = rek::primitives::Blob::new(input_file);

Finally, the blob is used to populate the Image parameter input for detect_labels(). The Image type is exported from the types sub-module. It uses the builder pattern to set input parameters.

let input_image = rek::types::Image::builder().bytes(input_blob).build();

Now that we have the actual Image object defined, we can call the detect_labels() API. Because the SDK is async, we need to ensure that we tack on the await keyword at the end. Notice that we use the send() method to finalize the API call, which turns it into a Rust future.

let rek_result = rek_client::detect_labels().image(input_image).send().await;

After this line runs, the rek_result variable will be populated with the results (or error) from the Amazon Rekognition service. The specific data type wrapped by the Result<T, E> type is the DetectLabelsOutput type.

Exploring Rekognition Results

Now that we’ve retrieved label results from Amazon Rekognition, let’s explore the results.

When you detect labels on an image, you will typically get many results. For example, a car could be classified as other terms, such as a vehicle, truck, automobile, sedan, SUV, minivan, and so on. We will need to use a looping construct to iterate through each label that was returned from the Rekognition service.

Before we do that, we’ll check to ensure that no error occurred, by using the Rust Result type’s is_ok() method.

if rek_result.is_ok() {

  println!("Rekognition result was successful.");

}

This simple snippet will print some static text if the detect_labels() API call was successful.

Now let’s unwrap the actual results and iterate over them. After the println statement above, plug in the following code. We have to unwrap the result, but we also have to unwrap the labels, because it is wrapped by the Option<T> type.

for current_result in rek_result.unwrap().labels.unwrap() {

  println!("{}", current_result.name.unwrap());

}

As we iterate through the labels, we also need to unwrap the name property, as it wraps the underlying String value in the Option<T> type as well.

Each of the labels that is detected also includes a confidence score. This indicates the probability that the model has correctly detected the corresponding label, as a percentage.

If you’d like to print out the confidence score, you could update the println statement to the below example.

println!("{} {}", current_result.name.unwrap(), current_result.confidence.unwrap());

Finally, run your program by using the cargo run command.

Congratulations! You’ve successfully used the Amazon Rekognition service to detect labels (descriptors) of an image file.

Project Sample Code

You can find the example code for this project on the StratusGrid GitHub organization.

Next Steps for Enhancing Your App

Now that you understand how to detect labels in image files, using Rust, you can expand this simple solution to do a lot more! Here are some ideas for other tasks you could perform, in relation to the Amazon Rekognition service:

• Iterate across all files within a directory, and perform label detection on each
• Explore the other Rekognition APIs, besides detect_labels
• Store the recognized label results in a database, such as Amazon DynamoDB or RDS
• Use S3 event triggers to run Rekognition APIs against image files, using Rust in AWS Lambda

StratusGrid is Your Partner in Smarter Cloud Solutions

At StratusGrid, we understand the complexities and potential of advanced label detection applications like the one you've just explored in our blog. Specializing in leveraging the power of Rust and Amazon Rekognition, we provide bespoke solutions and expert guidance to transform your projects. Whether you're doing image processing, fine-tuning performance for real-time detection, or seeking innovative ways to integrate AI into your applications, our team of seasoned AWS and Rust experts is here to move your vision forward.

Contact StratusGrid today – let's collaborate to turn your label detection aspirations into groundbreaking realities.

BONUS: Find Out if Your Organization is Ready for The Cloud ⤵️