Rusoto

Rusoto is an AWS SDK for Rust.

Rusoto consists of one core crate, containing common functionality shared across services, and then a crate for each supported service.

Services are generated from the botocore project's API definitions.

Rusoto also provides a credential crate. The credential crate provides the necessary functionality for properly loading and handling AWS credentials.

Help

If you are experiencing problems with rusoto, or would like clarification regarding something, please don't hesitate to contact us. You can reach us by opening an issue on the GitHub page for the project with the relevant information (code snippets and error messages are very helpful!). Alternatively, you can join us on Discord.

Usage & Example

Rusoto provides a crate for each AWS service it supports, containing a client and all of the associated types for that service. A full list of these services can be found on the Supported AWS Services page.

It also provides a core crate called rusoto_core, containing all shared functionality across services, such as the list of regions, signed request senders, and credential loading.

Consult the rustdoc documentatation by running cargo doc or visiting the online API documentation for the latest crates.io release.

An example of using Rusoto's DynamoDB API to list the names of all the tables in a database:

Cargo.toml:

[package]
name = "my-crate"
version = "0.1.0"
authors = ["My Name <my@email.com>"]

[dependencies]
rusoto_core = "0.42"
rusoto_dynamodb = "0.42"

Rust code:

extern crate rusoto_core;
extern crate rusoto_dynamodb;

use std::default::Default;

use rusoto_core::Region;
use rusoto_dynamodb::{DynamoDb, DynamoDbClient, ListTablesInput};

fn main() {
  let client = DynamoDbClient::new(Region::UsEast1);
  let list_tables_input: ListTablesInput = Default::default();

  match client.list_tables(list_tables_input).sync() {
    Ok(output) => {
      match output.table_names {
        Some(table_name_list) => {
          println!("Tables in database:");

          for table_name in table_name_list {
            println!("{}", table_name);
          }
        }
        None => println!("No tables in database!"),
      }
    }
    Err(error) => {
      println!("Error: {:?}", error);
    }
  }
}

Crates that extend Rusoto

Rusoto's current focus is to support all AWS services. This means higher level functionality present in other AWS SDKs are not currently available in Rusoto. See https://github.com/rusoto/rusoto/issues/1098 for more information on where Rusoto efforts are currently focused.

While Rusoto doesn't support various higher level functionality or better ergonomics, there are third party crates available that use Rusoto and provide this behavior.

A non-exhaustive list of projects that build on Rusoto to make it more ergonomic or provide higher level functionality:

1. dynomite: make your rust types fit dynamodb and visa versa
2. envy-store: deserialize AWS Parameter Store values into typesafe structs
3. S4: Simpler Simple Storage Service for Rust

If you'd like to add an example to this list, please open an issue or pull request on the rusoto.org repo.

Lambdas

Lambdas are a great place to use Rusoto! The runtime performance keeps costs down while the Rust language prevents classes of bugs found in other common languages used with AWS Lambda.

Building for Lambda

Lambda environments are x86 64 bit Linux. Existing Rust code can be cross-compiled from a different platform for this target. For example, on a Macbook, one can compile a Linux executable for Lambda. See Creating my first AWS Lambda using Rust for more information on setup.

Rustls

Rustls is modern TLS library written in Rust. It can replace OpenSSL in many places, including Rusoto. This makes it far easier to get a working lambda executable. No more The OpenSSL library reported an error on lambda startup because an OpenSSL flag isn't set right!

The rustls option must be set on the project's Cargo.toml file for both the rusoto_core crate and any Rusoto service crates used. For example, to use rustls with S3 and SQS:

rusoto_core = {version = "0.42.0", default_features = false, features=["rustls"]}
rusoto_s3 = {version = "0.42.0", default_features = false, features=["rustls"]}
rusoto_sqs = {version = "0.42.0", default_features = false, features=["rustls"]}

The default features must be disabled since that uses the hyper-tls crate. These flags can be found in the rusoto_core Cargo.toml.

Regions

Rusoto supports all regions AWS supports. There is also a Default implementation that reads from environment variables AWS_DEFAULT_REGION or AWS_REGION. For more information and to see the list of regions, see the Region API documentation.

Custom Regions

Custom Regions can be used for interacting with services that behave like AWS services, such as Ceph, Minio and local DynamoDB.

For a complete example of using local DynamoDB, see this sample project.

extern crate rusoto_core;
extern crate rusoto_dynamodb;

use rusoto_core::Region;
use rusoto_dynamodb::{DynamoDb, DynamoDbClient, ListTablesInput};

fn main() {
    // Create custom Region
    let region = Region::Custom {
        name: "us-east-1".to_owned(),
        endpoint: "http://localhost:8000".to_owned(),
    };

    let client = DynamoDbClient::new(region);

    let list_tables_request = ListTablesInput::default();
    let tables = client.list_tables(list_tables_request).sync();

    println!("Tables found: {:?}", tables);
}

Using Rusoto Futures

Rusoto Futures are like other Futures in Rust. They implement the Future trait:

Impl Future for RusotoFuture

Treating all calls to AWS services as asynchronous allows for different ways to handle network calls, potentially improving throughput and resource usage.

All API calls return a future

All Rusoto calls to AWS return a Future. This Future can be acted on immediately, saved for later use or chained/combined with other Futures before running.

.sync()

Rusoto supports returning Futures to be executed later as well as synchronous, blocking calls. The .sync() function on Rusoto Futures allows this behavior.

Under the hood it uses a tokio::runtime to immediately run the Future..

Chaining futures together

The usual collection of Future combinators can be used on Rusoto Futures. To read more about combinators, see the tokio docs on Futures. Another resource to help sort out compilation errors with Futures is the Futures Cheatsheet.

The example below uses the .then() combinator to chain multiple Futures together.

Examples

See Chaining Rusoto Futures blog post for more information and links to complete examples.

extern crate futures;
extern crate rusoto_core;
extern crate rusoto_dynamodb;
extern crate tokio_core;

use futures::future::Future;
use rusoto_core::{Region, RusotoError};
use rusoto_dynamodb::{
    AttributeDefinition, AttributeValue, CreateTableInput, CreateTableOutput, DynamoDb,
    DynamoDbClient, GetItemError, GetItemInput, GetItemOutput, KeySchemaElement,
    UpdateItemInput, UpdateItemOutput,
};
use std::collections::HashMap;
use tokio_core::reactor::Core;

fn main() {
    let item = make_item();
    let client = get_dynamodb_local_client();
    let mut core = Core::new().unwrap();

    let create_table_future = make_create_table_future(&client);
    let upsert_item_future = make_upsert_item_future(&client, &item);
    let item_from_dynamo_future = make_get_item_future(&client, &item);

    let chained_futures = create_table_future
        .then(|_| upsert_item_future)
        .then(|_| item_from_dynamo_future);

    let item_from_dynamo = match core.run(chained_futures) {
        Ok(item) => item,
        Err(e) => panic!("Error completing futures: {}", e),
    };
    println!("item: {:?}", item_from_dynamo);
}

fn make_create_table_future(client: &DynamoDbClient) -> 
  impl Future<Item = CreateTableOutput> {
    let attribute_def = AttributeDefinition {
        attribute_name: "foo_name".to_string(),
        attribute_type: "S".to_string(),
    };
    let k_schema = KeySchemaElement {
        attribute_name: "foo_name".to_string(),
        key_type: "HASH".to_string(), // case sensitive
    };
    let make_table_request = CreateTableInput {
        table_name: "a-testing-table".to_string(),
        attribute_definitions: vec![attribute_def],
        key_schema: vec![k_schema],
        ..Default::default()
    };

    client.create_table(make_table_request)
}

fn make_upsert_item_future(
    client: &DynamoDbClient,
    item: &HashMap<String, AttributeValue>,
) -> impl Future<Item = UpdateItemOutput> {
    let add_item = UpdateItemInput {
        key: item.clone(),
        table_name: "a-testing-table".to_string(),
        ..Default::default()
    };

    client.update_item(add_item)
}

fn make_get_item_future(
    client: &DynamoDbClient,
    item: &HashMap<String, AttributeValue>,
) -> impl Future<Item = GetItemOutput, Error = RusotoError<GetItemError>> {
    // future for getting the entry
    let get_item_request = GetItemInput {
        key: item.clone(),
        table_name: "a-testing-table".to_string(),
        ..Default::default()
    };
    client.get_item(get_item_request)
}

fn make_item() -> HashMap<String, AttributeValue> {
    let item_key = "foo_name";
    let mut item = HashMap::new();
    item.insert(
        item_key.to_string(),
        AttributeValue {
            s: Some("baz".to_string()),
            ..Default::default()
        },
    );

    item
}

fn get_dynamodb_local_client() -> DynamoDbClient {
    // Create custom Region
    let region = Region::Custom {
        name: "us-east-1".to_owned(),
        endpoint: "http://localhost:8000".to_owned(),
    };

    DynamoDbClient::new(region)
}

Disabling SSL Certificate Check

⚠️ Danger ⚠️

Disabling certificate checking is not part of regular use of Rusoto. There are very few legitimate reasons to disable default behavior. Production use of Rusoto should not use the behavior documented on this page. Use of AWS services should not disable certificate checking.

Use this only if you know what you're doing.

Opting out of certificate checking

The process of disabling an SSL cert check requires a few steps before using the services in Rusoto. The general steps are as follows:

1. Create a TlsConnector using the TlsConnectorBuilder. Ensure the certificate check has been disabled. This is the important part!
2. Create a HttpConnector, this is needed to implement the From trait on the HttpsConnector.
3. Create a HttpsConnector using from, passing in the HTTP and TLS connectors.
4. Create a Rusoto HttpClient using the from_connector method.
5. Initialize a service using the new_with method.
6. Build Things!

extern crate hyper;
extern crate hyper_tls;
extern crate native_tls;
extern crate rusoto_core;
extern crate rusoto_s3;

use hyper::client::HttpConnector;
use hyper_tls::HttpsConnector;
use native_tls::TlsConnector;
use rusoto_core::{DefaultCredentialsProvider, HttpClient, Region};
use rusoto_s3::{S3Client, S3};

fn main() {
    // 1. create tls connector which accepts invalid certs
    let tls_connector: TlsConnector = TlsConnector::builder()
        .danger_accept_invalid_certs(true)
        .build()
        .expect("failed to build tls connector");

    // 2. create http connector - make sure to not enforce http
    let mut http_connector = HttpConnector::new(4);
    http_connector.enforce_http(false);

    // 3. create https connector
    let https_connector = HttpsConnector::from((http_connector, tls_connector));

    // 4. create rusoto http client to be used in rusoto services
    let http_client = HttpClient::from_connector(https_connector);

    let cred_provider = DefaultCredentialsProvider::new().expect("failed to create cred provider");

    // custom region to talk to local service with a self-signed cert
    let local_region = Region::Custom {
        name: "us-east-1".to_owned(),
        endpoint: "https://localhost:8000".to_owned(),
    };
    // 5. initialize service clients with new http client
    let s3_client = S3Client::new_with(http_client, cred_provider, local_region);
    // 6. build!
    s3_client
        .list_buckets()
        .sync()
        .expect("failed to list buckets");
}

⚠️ Danger ⚠️

Do not use this for communicating to AWS services. Use this only for AWS-like services such as Ceph or Minio and generating trusted certificates is not possible.

Supported AWS Services

Debugging

Rusoto uses the log logging facade. For tests, Rusoto uses env_logger.

In order to see logging output (e.g. the actual requests made to AWS), env_logger needs to be initialized:

extern crate rusoto_core;
extern crate rusoto_s3;
extern crate env_logger;

use std::default::Default;

use rusoto_core::Region;
use rusoto_s3::{S3, S3Client, ListObjectsRequest};

fn main() {
    let _ = env_logger::try_init(); // This initializes the `env_logger`

    let bare_s3 = S3Client::new(Region::UsWest2);

    let mut list_request = ListObjectsRequest::default();
    list_request.bucket = "rusototester".to_string();
    let result = bare_s3.list_objects(list_request).sync().unwrap();
    println!("result is {:?}", result);
}

To see the output of logging from integration tests, the command needs to be run as follows:

RUST_LOG=rusoto,hyper=debug cargo test --features all

To get the logging output as well as the output of any println! statements, run:

RUST_LOG=rusoto,hyper=debug cargo test --features all -- --nocapture

If more debugging is required, all debug info including details from the compiler can be seen by setting RUST_LOG to debug. This will be noisy but will give a lot of debug information. For example:

RUST_LOG=debug cargo test --features all

Performance

Currently, performance isn't benchmarked.

To get the best runtime performance in a project using Rusoto, compile with the --release flag.

For example: cargo build --release

Increasing performance with large objects

As of Rusoto 0.36.0, we expose the http1_read_buf_exact_size setting in the hyper HTTP client. This can greatly improve performance dealing with payloads greater than 100MB. See https://github.com/rusoto/rusoto/pull/1227 for more information.

Example of increasing the buffer size from the default of 8KB to 2MB:

extern crate rusoto_core;
extern crate rusoto_ecs;

use rusoto_ecs::{Ecs, EcsClient, ListClustersRequest};
use rusoto_core::{Region, DefaultCredentialsProvider};
use rusoto_core::request::{HttpClient, HttpConfig};

fn main() {
    // EcsClient configuration demonstrates setting the hyper read_buf_size option
    // to 2MB:
    let cred_provider =  DefaultCredentialsProvider::new().unwrap();
    let mut http_config_with_bigger_buffer = HttpConfig::new();
    http_config_with_bigger_buffer.read_buf_size(1024 * 1024 * 2);
    let http_provider = HttpClient::new_with_config(http_config_with_bigger_buffer).unwrap();

    let ecs = EcsClient::new_with(http_provider, cred_provider, Region::UsEast1);

    match ecs.list_clusters(ListClustersRequest::default()).sync() {
        Ok(clusters) => {
            for arn in clusters.cluster_arns.unwrap_or(vec![]) {
                println!("arn -> {:?}", arn);
            }
        }
        Err(err) => {
            panic!("Error listing container instances {:#?}", err);
        }
    }
}

Potential runtime performance improvements

• Improve XML parsing

Migrations

Rusoto Versions < 0.38.0 to Versions >= 0.38.0 Error Handling

Rusoto version 0.38.0 introduced the enum rusoto_core::RusotoError. In versions before 0.38.0 errors would be typed as themselves such as rusoto_s3::CreateBucketError. In versions >= 0.38.0 the service method errors would be wrapped in the new enum, such as rusoto_core::RusotoError<rusoto_s3::CreateBucketError>.

Rusoto < 0.38.0:

extern crate rusoto_core;
extern crate rusoto_s3;

use rusoto_core::Region;
use rusoto_s3::{CreateBucketError, CreateBucketOutput, CreateBucketRequest, S3Client, S3};

fn main() {
    let s3_client = S3Client::new(Region::UsEast1);
    let bucket_name = "itshabib-buckets".to_owned();

    match create_bucket_sync(&s3_client, bucket_name.clone()) {
        Ok(res) => println!("successfully created bucket! resp: {:#?}", res),
        Err(err) => println!("Error creating bucket. err: {:#?}", err),
    };
}


// error handling for versions < 0.38
fn create_bucket_sync(
    s3_client: &S3Client,
    bucket_name: String,
    // errors are not wrapped by RusotoError in version < 0.38
) -> Result<CreateBucketOutput, CreateBucketError> {
    let create_bucket_req = CreateBucketRequest {
        bucket: bucket_name.clone(),
        ..Default::default()
    };

    s3_client.create_bucket(create_bucket_req).sync()
}

Rusoto >= 0.38.0:

extern crate rusoto_core;
extern crate rusoto_s3;

use rusoto_core::{Region, RusotoError};
use rusoto_s3::{CreateBucketError, CreateBucketOutput, CreateBucketRequest, S3Client, S3};

fn main() {
    let s3_client = S3Client::new(Region::UsEast1);
    let bucket_name = "itshabib-buckets".to_owned();

    match create_bucket_sync(&s3_client, bucket_name.clone()) {
        Ok(res) => println!("successfully created bucket! resp: {:#?}", res),
        Err(err) => println!("Error creating bucket. err: {:#?}", err),
    };
}

// error handling for versions >= 0.38
fn create_bucket_sync(
    s3_client: &S3Client,
    bucket_name: String,
    // service method error is wrapped in RusotoError enum
) -> Result<CreateBucketOutput, RusotoError<CreateBucketError>> {
    let create_bucket_req = CreateBucketRequest {
        bucket: bucket_name.clone(),
        ..Default::default()
    };

    s3_client.create_bucket(create_bucket_req).sync()
}

Rusoto 0.31.0 to 0.32.0

With async support, the easiest migration is to use the new simple() constructor on clients and add .sync() to the function calls.

Rusoto 0.31.0:

extern crate rusoto_core;
extern crate rusoto_polly;

use rusoto_polly::{Polly, PollyClient, DescribeVoicesInput};
use rusoto_core::{DefaultCredentialsProvider, Region, default_tls_client};

...
let credentials = DefaultCredentialsProvider::new().unwrap();
let client = PollyClient::new(default_tls_client().unwrap(), credentials, Region::UsEast1);
let request = DescribeVoicesInput::default();


println!("{:?}", client.describe_voices(&request).unwrap());

Rusoto 0.32.0 with async:

extern crate rusoto_core;
extern crate rusoto_polly;

use rusoto_polly::{Polly, PollyClient, DescribeVoicesInput};
use rusoto_core::Region;

...
let client = PollyClient::simple(Region::UsEast1);
let request = DescribeVoicesInput::default();

println!("{:?}", client.describe_voices(&request).sync().unwrap());

Note:

• DefaultCredentialsProvider isn't required with the ::simple() constructor
• default_tls_client isn't required with the ::simple() constructor
• .sync() is called on the result from describe_voices. This blocks until the call is finished.

Rusoto 0.24.0 to 0.25.0 or later

As of Rusoto 0.25.0, the Rusoto crate is now deprecated. This decision was made because the single crate implementing all AWS services was too large to compile, especially on TravisCI and Appveyor. The new main crate is rusoto_core and all services now have their own crate.

To continue implementing new services and reducing compilation times, we've extracted a few core crates and every AWS service we support now has its own crate.

rusoto_core now contains the core functionality of Rusoto: AWS signature handling, regions, requests to services and XML helpers.

rusoto_mock was also extracted. Crate users shouldn't need this: it's for developing on Rusoto itself.

rusoto_credential remains its own crate, providing AWS credential sourcing. If you're working on something that uses AWS and Rusoto doesn't support it, you can use that crate instead of rolling your own AWS credential providers.

The new service crates depend on rusoto_core.

Required changes

Previously, to bring in a Rusoto implementation of an AWS service, you'd specify something like this in your Cargo.toml file:

rusoto = {version = "0.24", features = ["rds"]}

Now, you'd bring in services like this:

rusoto_core = {version = "0.25.0"}
rusoto_rds = {version = "0.25.0"}

Once the new crates have been brought in, use the new crates in your code. A sample before:

extern crate rusoto;

use rusoto::rds::{RdsClient, CreateDBInstanceMessage, DescribeDBInstancesMessage};
use rusoto::{DefaultCredentialsProvider, Region, default_tls_client};

And after:

extern crate rusoto_core;
extern crate rusoto_rds;
use rusoto_rds::{Rds, RdsClient, CreateDBInstanceMessage, DescribeDBInstancesMessage};
use rusoto_core::{DefaultCredentialsProvider, Region, default_tls_client};

Note there are now two crates required: rusoto_core as well as the RDS crate, rusoto_rds. There's also a new trait for each service. In this case it's Rds and we bring that in. This is used to make calls to services easier to test and improve ergonomics of using Rusoto clients.

Credential timeouts

If AWS credentials aren't found locally, Rusoto attempts to query the ECS container role provider, then the EC2 Instance Profile provider. When running locally, such as in a command line tool, timing out waiting for those services can take 30 seconds or more.

To improve responsiveness of credential sourcing, timeouts can be provided to the credential provider. This example uses a 200ms timeout:

extern crate rusoto_core;
extern crate rusoto_s3;

use rusoto_core::credential::ChainProvider;
use rusoto_core::request::HttpClient;
use rusoto_core::Region;
use rusoto_s3::{S3, S3Client};
use std::time::{Duration, Instant};

fn main() {
    let mut chain = ChainProvider::new();
    chain.set_timeout(Duration::from_millis(200));
    let s3client = S3Client::new_with(
        HttpClient::new().expect("failed to create request dispatcher"),
        chain,
        Region::UsEast1,
    );

    let start = Instant::now();
    println!("Starting up at {:?}", start);

    match s3client.list_buckets().sync() {
        Err(e) => println!("Error listing buckets: {}", e),
        Ok(buckets) => println!("Buckets found: {:?}", buckets),
    };
    println!("Took {:?}", Instant::now().duration_since(start));
}

Now, if Rusoto can't find any credentials within 200ms, it will return an error like this:

Error listing buckets: Couldn't find AWS credentials in environment, credentials file, or IAM role.

Last updated for Rusoto 0.34.0 with rusoto_credential 0.13.0.