Using Amazon SQS from Rust Applications

Learn how to use Simple Queue Service (AWS SQS) with Rust to create, send, receive, and delete messages from queues. Contact StratusGrid for help.

Subscribe

Subscribe

How to Use AWS SQS from Rust Applications | StratusGrid
9:47

Amazon Simple Queueing Service (SQS) is a managed service that provides message queues.

You can easily create a message queue, send messages into the queue from a “producer” application, and then read & process those messages from a “consumer” application. Using message queues helps to decouple application components from each other, so that each component can be scaled independently, according to its need.

 

There are many open source alternatives to SQS such as RabbitMQ, ZeroMQ, BlazingMQ, Apache ActiveMQ, and more. Each of these message queueing solutions has a unique set of features and deployment model, so be sure to check out their documentation.

Message queues can receive messages from multiple producers, and multiple consumers can pull messages off the queue. This is what allows each component to scale independently. The structure of a “message” is arbitrary, and the responsibility of an application developer.

A message might simply contain unstructured text, or a structured JSON or YAML object, or some other format.

You can create many different queues in SQS, depending on the purpose of the application. Generally speaking, you’ll want to create a separate queue for each integration between different services.

Common API operations you’ll encounter, when working with Amazon SQS queues include:

  • Creating an SQS queue
  • Sending a message to an SQS queue
  • Receiving a message from an SQS queue
  • Deleting a message from an SQS queue
  • Deleting an SQS queue

You’ll notice that Amazon SQS supports a wide variety of APIs beyond the basics, for more advanced use cases, such as sending or receiving messages in batches. We’ll explore how you can use the AWS SDK for Rust to utilize Amazon SQS queues for the most fundamental operations.

Prerequisites

Before we get started, make sure that you have the prerequisites taken care of.

Once the Rust toolchain is installed, you should have access to tools such as the Rust compiler (rustc), and the cargo package manager.

Create Rust Project

Let’s create a brand new Rust project, using the cargo CLI tool. Open up your terminal and run these commands:

cargo new sqs-test

cd sqs-test

Next, we’ll need to install our dependencies into the project. Rust packages are called “crates” and are commonly hosted on the publicly accessible crates.io registry. The AWS SDK for Rust uses async Rust, so we’ll need to install an async executor. Tokio is the most common async runtime, so we’ll use that.

cargo add tokio --features=full

We also need to install the aws-config base crate, along with the crate specifically for Amazon SQS.

cargo add aws-config --features=behavior-version-latest

cargo add aws-sdk-sqs

Another crate we’ll add allows us to load our AWS credentials from a separate environment variable file. This prevents us from hard-coding our credentials into our Rust source code files. This crate is called dotenvy.

cargo add dotenvy

Now that we’ve installed the necessary crates, it’s time to set up our AWS credentials for SQS.

Setup AWS IAM User

As with any local application that utilizes AWS APIs, we need to create an IAM user with an AWS access key to authenticate.

  • Open the AWS Management Console
  • Navigate to the AWS IAM service
  • Create a new IAM User
  • Attach the AmazonSQSFullAccess policy

After the user has been created, generate a new access key.

  • Open the User Details page
  • Select the Security Credentials tab
  • Click the Create Access Key button
  • Click Other, then Next
  • Click the Create Access Key button

Copy the access key ID and secret access key into a new file named .env in the root of your project directory.

AWS_ACCESS_KEY_ID=<PASTE_HERE>

AWS_SECRET_ACCESS_KEY=<PASTE_HERE>
AWS_REGION=us-west-1

Configure the AWS SDK

Now that you’ve installed the necessary creates and set your credentials, it’s time to set up your async main function. Open up your src/main.rs file and add the following main function.

#[tokio::main]

async fn main() {

    dotenvy::dotenv();

    let sdkcfg = aws_config::load_from_env().await;

}

This code will run your main function using the Tokio async executor, load the environment variables you’ve declared in your .env file, and then create a new AWS SDK configuration from the environment.

Now we’re ready to work with the Amazon SQS APIs. Let’s start by creating an Amazon SQS client, using the SDK configuration. Add the following line after the last line in the main function.

let sqs_client = aws_sdk_sqs::Client::new(&sdkcfg);

Now that you’ve created the SQS client object, you can access the various API actions available in the SQS service.

Create an SQS Queue

Let’s start by programmatically creating a new SQS queue, using the create_queue() method. This method returns a CreateQueueFluentBuilder object, which defines several methods that allow you to configure the request before sending it. The only required input to create a queue is a queue name, so we’ll specify that.

sqs_client.create_queue()

    .queue_name("rust-test123")

    .send().await;

After configuring the request, you’ll need to call send() on the builder, and then await the future, in order for the API call to actually be sent.

If you run the program as it currently is, you should see a new queue created.

cargo run

You don’t strictly have to capture the result into a variable, but generally you should do so, to determine if the request was successful or not. Let’s adjust that now.

let sqs_result = sqs_client.create_queue()

    .queue_name("rust-test123")

    .send().await;




match sqs_result {

    Ok(result) => {

        println!("Successfully created queue. New queue URL is: {0}", result.queue_url.unwrap());

    }

    Err(err) => {

        println!("An error occurred while creating the queue: {0:?}", err);

    }

}

This code uses the Rust match statement to test if the result was successful or not, and print out a debugging statement accordingly.

Send Message to SQS Queue

Now that you’ve successfully created an SQS queue using Rust, let’s learn how to send a message to a queue. This time we’ll use the send_message method on the SQS client struct to send a message to a specific queue.

You’ll need to know the Queue URL, which is returned when you create a new SQS queue, or you can retrieve it from the AWS Management Console. You can easily construct the queue URL by substituting your queue name, region, and AWS account ID in the appropriate spots.

let queue_url = "https://sqs.us-west-1.amazonaws.com/973081273628/rust-test123";

let send_result = sqs_client.send_message()

    .queue_url(queue_url)

    .send().await;

To set the queue URL, use the queue_url method on the SendMessageFluentBuilder. Let’s see what happens if we try to run this, but first we’ll add a success and error handler, in the form of another match statement.

match send_result {

    Ok(result) => {

        println!("Successfully sent message to Amazon SQS queue {0}", result.message_id.unwrap_or_default());

    }

    Err(err) => {

        println!("Error occurred while sending message {0:?}", err);

    }

}

If you try to run this code with cargo run, you should see an error message indicating that the message body is empty, and must be included. 

The request must contain the parameter MessageBody.”

Let’s adjust the API call to include a message body, using the message_body method.

let send_result = sqs_client.send_message()

    .queue_url(queue_url)

    .message_body("My favorite food is bacon")

    .send().await;

Try running the program again. As you can see, a message is successfully sent into the message queue. The key learning takeaway here is that you’ll need to ensure that all of the necessary input parameters are provided to the API call, in order to call it successfully.

Each AWS API has a different set of required parameters, so refer to the documentation for the API you’re trying to call.

Receive Message from SQS Queue

Now that the SQS queue has a message, it’s time to learn how to receive a message from the queue, using Rust. Rather than calling send_message, we’ll call the receive_message method on the SQS client struct.

To receive a message from an SQS queue, we once again need to specify the queue URL, just like when we sent a message to the queue.

let receive_result = sqs_client.receive_message()

    .queue_url(queue_url)

    .send().await;

After receiving messages from SQS, you will need to iterate through the results. In response, you could get zero messages, one message, or multiple messages. Hence, you’ll need to check to see if there are any results or not.

We can do that by using the if..let construct in Rust. If there’s one or more messages, then we’ll use a simple for loop to iterate through them, and print out the message body.

match receive_result {

    Ok(msgs) => {

        if let Some(msgs) = msgs.messages {

            for msg in msgs {

                println!("Message received: {0}", msg.body.unwrap_or_default());

            } 

        }

    }

    Err(err) => {

        println!("Error occurred while receiving message {0:?}", err);

    }

}

After you receive a message, you also must delete the message off the queue, using the delete_message method, once processing has finished successfully. Each message that’s received from an SQS queue is automatically assigned a unique ID called a “receipt handle.” You can use this receipt handle to delete the message from the queue.

IMPORTANT: If you do not delete the message from the queue explicitly, before the Visibility Timeout period, SQS will put the message back onto the queue for re-processing. This default behavior helps to ensure reliable message processing.

To delete these messages, let’s add some code to call the delete_message method during each iteration over the results of the receive call. Add the following code after the println statement that prints out the message body.

let delete_result = sqs_client.delete_message()

    .queue_url(queue_url)

    .receipt_handle(msg.receipt_handle.unwrap_or_default())

    .send().await;

if delete_result.is_ok() {

    println!("Deleted message from queue");

}

As you can see, we call delete_message, specify the same queue URL, and pass in the receipt handle for each message. Rather than write out a full match statement, we can just check the result of the is_ok function to see if the deletion was successful or not.

Building Decoupled Services with SQS and Rust

Now that you understand how to use SQS queues from Rust applications, you can begin building decoupled services. Any services that require heavy data processing should be developed and scaled independently from other services with lesser requirements. Use message queues to queue up work for the service that handles the heavy processing.

A common example of decoupled services could be uploading and processing video files. The service that handles ingestion of video files could be fairly lightweight, whereas the service that actually performs video processing tasks could be much heavier.

Check out the StratusGrid YouTube channel for more information on AWS and Rust!

Get Expert Help with Your AWS SQS Integrations

Need assistance with your Amazon SQS implementations? Contact StratusGrid for a consultation.

Our team of AWS specialists can help you seamlessly integrate SQS into your applications, optimize your message processing, and ensure your systems are scalable and efficient.

Reach out to StratusGrid today and take your cloud solutions to the next level.

BONUS: Find Out if Your Organization is Ready for The Cloud ⤵️

FinOps

Similar posts