Learn how to use Simple Queue Service (AWS SQS) with Rust to create, send, receive, and delete messages from queues. Contact StratusGrid for help.
Amazon Simple Queueing Service (SQS) is a managed service that provides message queues.
You can easily create a message queue, send messages into the queue from a “producer” application, and then read & process those messages from a “consumer” application. Using message queues helps to decouple application components from each other, so that each component can be scaled independently, according to its need.
There are many open source alternatives to SQS such as RabbitMQ, ZeroMQ, BlazingMQ, Apache ActiveMQ, and more. Each of these message queueing solutions has a unique set of features and deployment model, so be sure to check out their documentation.
Message queues can receive messages from multiple producers, and multiple consumers can pull messages off the queue. This is what allows each component to scale independently. The structure of a “message” is arbitrary, and the responsibility of an application developer.
A message might simply contain unstructured text, or a structured JSON or YAML object, or some other format.
You can create many different queues in SQS, depending on the purpose of the application. Generally speaking, you’ll want to create a separate queue for each integration between different services.
Common API operations you’ll encounter, when working with Amazon SQS queues include:
You’ll notice that Amazon SQS supports a wide variety of APIs beyond the basics, for more advanced use cases, such as sending or receiving messages in batches. We’ll explore how you can use the AWS SDK for Rust to utilize Amazon SQS queues for the most fundamental operations.
Before we get started, make sure that you have the prerequisites taken care of.
Once the Rust toolchain is installed, you should have access to tools such as the Rust compiler (rustc), and the cargo package manager.
Let’s create a brand new Rust project, using the cargo CLI tool. Open up your terminal and run these commands:
cargo new sqs-test
cd sqs-test
Next, we’ll need to install our dependencies into the project. Rust packages are called “crates” and are commonly hosted on the publicly accessible crates.io registry. The AWS SDK for Rust uses async Rust, so we’ll need to install an async executor. Tokio is the most common async runtime, so we’ll use that.
cargo add tokio --features=full
We also need to install the aws-config base crate, along with the crate specifically for Amazon SQS.
cargo add aws-config --features=behavior-version-latest
cargo add aws-sdk-sqs
Another crate we’ll add allows us to load our AWS credentials from a separate environment variable file. This prevents us from hard-coding our credentials into our Rust source code files. This crate is called dotenvy.
cargo add dotenvy
Now that we’ve installed the necessary crates, it’s time to set up our AWS credentials for SQS.
As with any local application that utilizes AWS APIs, we need to create an IAM user with an AWS access key to authenticate.
After the user has been created, generate a new access key.
Copy the access key ID and secret access key into a new file named .env in the root of your project directory.
AWS_ACCESS_KEY_ID=<PASTE_HERE>
AWS_SECRET_ACCESS_KEY=<PASTE_HERE>
AWS_REGION=us-west-1
Now that you’ve installed the necessary creates and set your credentials, it’s time to set up your async main function. Open up your src/main.rs file and add the following main function.
#[tokio::main]
async fn main() {
dotenvy::dotenv();
let sdkcfg = aws_config::load_from_env().await;
}
This code will run your main function using the Tokio async executor, load the environment variables you’ve declared in your .env file, and then create a new AWS SDK configuration from the environment.
Now we’re ready to work with the Amazon SQS APIs. Let’s start by creating an Amazon SQS client, using the SDK configuration. Add the following line after the last line in the main function.
let sqs_client = aws_sdk_sqs::Client::new(&sdkcfg);
Now that you’ve created the SQS client object, you can access the various API actions available in the SQS service.
Let’s start by programmatically creating a new SQS queue, using the create_queue() method. This method returns a CreateQueueFluentBuilder object, which defines several methods that allow you to configure the request before sending it. The only required input to create a queue is a queue name, so we’ll specify that.
sqs_client.create_queue()
.queue_name("rust-test123")
.send().await;
After configuring the request, you’ll need to call send() on the builder, and then await the future, in order for the API call to actually be sent.
If you run the program as it currently is, you should see a new queue created.
cargo run
You don’t strictly have to capture the result into a variable, but generally you should do so, to determine if the request was successful or not. Let’s adjust that now.
let sqs_result = sqs_client.create_queue()
.queue_name("rust-test123")
.send().await;
match sqs_result {
Ok(result) => {
println!("Successfully created queue. New queue URL is: {0}", result.queue_url.unwrap());
}
Err(err) => {
println!("An error occurred while creating the queue: {0:?}", err);
}
}
This code uses the Rust match statement to test if the result was successful or not, and print out a debugging statement accordingly.
Now that you’ve successfully created an SQS queue using Rust, let’s learn how to send a message to a queue. This time we’ll use the send_message method on the SQS client struct to send a message to a specific queue.
You’ll need to know the Queue URL, which is returned when you create a new SQS queue, or you can retrieve it from the AWS Management Console. You can easily construct the queue URL by substituting your queue name, region, and AWS account ID in the appropriate spots.
let queue_url = "https://sqs.us-west-1.amazonaws.com/973081273628/rust-test123";
let send_result = sqs_client.send_message()
.queue_url(queue_url)
.send().await;
To set the queue URL, use the queue_url method on the SendMessageFluentBuilder. Let’s see what happens if we try to run this, but first we’ll add a success and error handler, in the form of another match statement.
match send_result {
Ok(result) => {
println!("Successfully sent message to Amazon SQS queue {0}", result.message_id.unwrap_or_default());
}
Err(err) => {
println!("Error occurred while sending message {0:?}", err);
}
}
If you try to run this code with cargo run, you should see an error message indicating that the message body is empty, and must be included.
“The request must contain the parameter MessageBody.”
Let’s adjust the API call to include a message body, using the message_body method.
let send_result = sqs_client.send_message()
.queue_url(queue_url)
.message_body("My favorite food is bacon")
.send().await;
Try running the program again. As you can see, a message is successfully sent into the message queue. The key learning takeaway here is that you’ll need to ensure that all of the necessary input parameters are provided to the API call, in order to call it successfully.
Each AWS API has a different set of required parameters, so refer to the documentation for the API you’re trying to call.
Now that the SQS queue has a message, it’s time to learn how to receive a message from the queue, using Rust. Rather than calling send_message, we’ll call the receive_message method on the SQS client struct.
To receive a message from an SQS queue, we once again need to specify the queue URL, just like when we sent a message to the queue.
let receive_result = sqs_client.receive_message()
.queue_url(queue_url)
.send().await;
After receiving messages from SQS, you will need to iterate through the results. In response, you could get zero messages, one message, or multiple messages. Hence, you’ll need to check to see if there are any results or not.
We can do that by using the if..let construct in Rust. If there’s one or more messages, then we’ll use a simple for loop to iterate through them, and print out the message body.
match receive_result {
Ok(msgs) => {
if let Some(msgs) = msgs.messages {
for msg in msgs {
println!("Message received: {0}", msg.body.unwrap_or_default());
}
}
}
Err(err) => {
println!("Error occurred while receiving message {0:?}", err);
}
}
After you receive a message, you also must delete the message off the queue, using the delete_message method, once processing has finished successfully. Each message that’s received from an SQS queue is automatically assigned a unique ID called a “receipt handle.” You can use this receipt handle to delete the message from the queue.
IMPORTANT: If you do not delete the message from the queue explicitly, before the Visibility Timeout period, SQS will put the message back onto the queue for re-processing. This default behavior helps to ensure reliable message processing.
To delete these messages, let’s add some code to call the delete_message method during each iteration over the results of the receive call. Add the following code after the println statement that prints out the message body.
let delete_result = sqs_client.delete_message()
.queue_url(queue_url)
.receipt_handle(msg.receipt_handle.unwrap_or_default())
.send().await;
if delete_result.is_ok() {
println!("Deleted message from queue");
}
As you can see, we call delete_message, specify the same queue URL, and pass in the receipt handle for each message. Rather than write out a full match statement, we can just check the result of the is_ok function to see if the deletion was successful or not.
Now that you understand how to use SQS queues from Rust applications, you can begin building decoupled services. Any services that require heavy data processing should be developed and scaled independently from other services with lesser requirements. Use message queues to queue up work for the service that handles the heavy processing.
A common example of decoupled services could be uploading and processing video files. The service that handles ingestion of video files could be fairly lightweight, whereas the service that actually performs video processing tasks could be much heavier.
Check out the StratusGrid YouTube channel for more information on AWS and Rust!
Need assistance with your Amazon SQS implementations? Contact StratusGrid for a consultation.
Our team of AWS specialists can help you seamlessly integrate SQS into your applications, optimize your message processing, and ensure your systems are scalable and efficient.
Reach out to StratusGrid today and take your cloud solutions to the next level.
Streamline AWS cross-account messaging with StratusGrid's SQS Queues module, ensuring secure, efficient inter-account communication.
Use StratusGrid's IAM Role Cross-Account Trusting Terraform module to securely manage your AWS cross-account interactions ensuring security and...
Unlock powerful data querying capabilities with StratusGrid's Athena ALB Table Terraform Module. Transform AWS ALB logs into actionable insights...