crate-ci

In Rust, there is a tendency towards a lot of small crates. The challenge with that is the overhead of maintaining a crate, from managing issues to releases.

The goal of this guide is to help you successfully manage your Rust crates with minimal overhead.

If you need further help, feel free to reach out to us on gitter

Contributing

  • Find a mistake or out of date examples? Feel free to file an issue or create a PR.
  • Have a way to improve the CI for crates? Feel free to join the discussion

Managing PRs

The goal of this section is to help you streamline the process for reviewing and accepting PRs.

Testing

Pre-requisites:

  • Crate exists
  • cargo test runs relevant tests

Questions to Consider

Platforms to Support?

Target platforms are a combination of:

  • OS
  • CPU
  • 32-bit or 64-bit
  • (Linux) libc version
  • (Windows) MSys or MSVC

These platforms fall into several major categories

  • Desktop
  • "Embedded": Phones, Raspberry Pis, etc
  • Embedded: Those running no_std

If you are creating a library, the ideal situation is to support as many of these as possible.

If you are creating a program, it is mostly a matter of having at least one form of binary on the systems you care about. It would be nice to support more in case a contributor's development environment is different than yours.

Rust versions to Support?

Some clients might not be able to always use the latest release. Maintainers have taken different approaches to this

  • Don't support older versions
  • Support N releases back
  • Only require compiler upgrades on major or minor

In addition to numbered, releases, Rust has the following release labels

  • stable: The current release
  • beta: The next release
  • nightly: Bleeding edge with feature flags to enable "unstable" features
    • "unstable" from the perspective of API/behavior compatibility; this does not speak to whether it is runtime stable

Recommendation:

  • Support 2 releases back
  • Features requiring nightly are hidden behind a feature flag

Platform / Rust Coverage?

You can run your tests on every version of Rust and across every platform at the cost of slower feedback on your PRs and increasing load on your chosen CI service.

Recommendation:

  • Test all platforms on "2 releases back" and stable
  • Test the minimum platforms on beta and nightly

CI Triggers

Travis and related CIs by default trigger builds on

  • Commits to branches in the repo
  • PRs
  • Tags

Additionally, you can schedule builds to happen periodically. This can be useful to catch problems from new versions of Rust or your dependencies (if you don't commit your lock file). It can be discouraging to a contributor if the build fails on their PR for reasons unrelated to their change.

In contrast, we want to be respectful of the CI services we are getting for free and don't want to put unnecessary load on them.

So as a healthy medium, we recommend:

  • If you use nightly-only features: build frequently (~1 / week)
  • All else: build monthly (0 0 1 * * in cron syntax)

Notes:

  • If the service offers it (like Travis), you probably want it set to "Do not run if there has been a build in the last 24 hours" rather than "Always run".
  • Appveyor has put extra restrictions on scheuled builds.

Running Tests on PRs

There are several CI hosts that will help you run tests on your CI

See also example-base.

TravisCI

Supports:

Steps:

  1. Sign up for TravisCI
  2. View your repositories by clicking on your name
  3. Follow the steps for "Review and add" your organization, if needed
  4. "Sync account" if your repo isn't showing up
  5. Toggle the switch for your repo
  6. Add a .travis.yml to your repo

This is a good starting .travis.yml:

sudo: false
language: rust
rust:
- 1.22.0  # Oldest supported
- stable
- beta
- nightly

os:
- linux
- osx
- windows

install:
- if [[ "$TRAVIS_OS_NAME" == "windows" ]]; then choco install windows-sdk-10.0; fi # windows workaround (for now)

script:
- cargo check --verbose
- cargo test --verbose

cache:
  cargo: true

Highlights

  • sudo: false: Allows Travis to do some optimizations
  • os: Applies all tests to listed operating systems
  • install: Windows workaround to allow rust to run properly. Remove if you are not testing on Windows. Hopefully temporary as Travis CI just started Windows support as of 2018-10-11.
  • cargo check: Only needed if your project has a [[bin]]. Ensure that that builds type. check delivers faster turnaround than doing cargo build since we don't care about the build artifact.
  • cargo test: If your crate has features, considering at least re-running the tests with --no-default-features and --all-features, as appropriate.
  • cache:: Cache the cargo registry and build

Appveyor

Supports:

  • Windows

This is a good starting appveyor.yml:

environment:
  global:
    CHANNEL: stable
    #APPVEYOR_CACHE_SKIP_RESTORE: true  # Uncomment when caching causes problems

  matrix:
  # Oldest supported
  - TARGET: x86_64-pc-windows-msvc
    CHANNEL: 1.22.0
  - TARGET: x86_64-pc-windows-gnu
    CHANNEL: 1.22.0
  # Stable channel
  - TARGET: i686-pc-windows-msvc
    CHANNEL: stable
  - TARGET: i686-pc-windows-gnu
    CHANNEL: stable
  - TARGET: x86_64-pc-windows-msvc
    CHANNEL: stable
  - TARGET: x86_64-pc-windows-gnu
    CHANNEL: stable
  # Beta channel
  - TARGET: x86_64-pc-windows-msvc
    CHANNEL: beta
  - TARGET: x86_64-pc-windows-gnu
    CHANNEL: beta
  # Nightly channel
  - TARGET: x86_64-pc-windows-msvc
    CHANNEL: nightly
  - TARGET: x86_64-pc-windows-gnu
    CHANNEL: nightly

install:
- ps: >-
    $Env:PATH += ';C:\msys64\usr\bin'
- curl -sSf -o rustup-init.exe https://win.rustup.rs/
- rustup-init.exe -y --default-host %TARGET% --default-toolchain %CHANNEL%
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- rustc -Vv
- cargo -V

test_script:
- cargo check --verbose
- cargo test --verbose

cache:
- C:\Users\appveyor\.cargo\registry
- target

notifications:
- provider: Email
  on_build_success: false

# Building is done in the test phase, so we disable Appveyor's build phase.
build: false

Highlights:

  • install
    • Install rust using Curl (distributed with msys64)
    • Print tool versions for traceability

Style Enforcement

Pre-requisites:

rustfmt is the standard tool for automating code formatting. You can run it manually to clean up your code, integrate it into your text editor to do it automatically, or have it verify if your code is formatted correctly. That last role is what we want to leverage to reduce reduce the burden of handling PRs.

See also example-rustfmt.

Specifying Your Style

We recommend you use the default style as that will be most universal within the Rust ecosystem.

With that said, it would be beneficial to capture a snapshot of that style.

  • If users are on different versions of rustfmt with different defaults, it will help minimize conflicts.
  • If a new rustfmt is released with a new default, this could cause PRs to start failing, frustrating contributors.

You can lock down your style by running:

rustfmt --dump-default-config .rustfmt.toml

TravisCI

Unlike your tests, there is little value in running more than one job to check the style. We recommend running it on Travis rather than Appveyor because Travis supports your jobs running in parallel.

We'll be adding the following to your .travis.yml:

matrix:
  include:
  - env: RUSTFMT
    rust: 1.24.0  # `stable`: Locking down for consistent behavior
    install:
    - rustup component add rustfmt-preview
    script:
    - cargo fmt -- --write-mode=diff

Highlights:

  • matrix: include: is allowing us to define a complete one-off build job.
    • This will run in parallel to your tests, giving you quicker feedback.
    • No other job output will be in here, making it easier to see the results.
  • rust: 1.24.0: We run a specific version of Rust to get its version of rustfmt
    • Locking down to a specific version is helpful to avoid behavior changes, even if bug fixes, from breaking PRs.
  • env: RUSTFMT: This is purely here because Travis will put it in the job summary, making it easier to distinguish this job from others

Code Smells

Pre-requisites:

Warnings

Compiler warnings provide some basic checking for code smells.

There are many ways to check for warnings. We recommend the followig:

In each of your lib.rs, main.rs, and test .rs files:

#![warn(warnings)]

In .travis.yml:

matrix:
  include:
  - env: RUSTFLAGS="-D warnings"
    rust: 1.24.0  # `stable`: Locking down for consistent behavior
    script:
    - cargo check --tests

Highlights

  • Doesn't slow people down during prototyping by turning warnings into errors.
  • Contributors see all warnings they will be accountable for due to warn(warnings).
  • Warning changes don't break the CI due to rust: 1.24.0

A major downside of this:

  • Only run on one target

See also example-warn.

Why Avoid The Simple Answer

The seemingly easy answer for checking for warnings is to either:

RUSTFLAGS="-D warnings" cargo build

or in each of your lib.rs, main.rs, and test .rs files:


# #![allow(unused_variables)]
#![deny(warnings)]
#fn main() {
#}

The reason you don't want to do this in your CI process is that new versions of Rust can add and remove warnings, causing the build to break on your contributor's PR, frustrating and possibly alienating them.

Lints

Linters are extra tools for checking for code smells. They aren't regular warnings either because

  • Slower to analyze
  • False positives

clippy is the standard linter for the Rust ecosystem.

See also example-clippy.

TravisCI

Like with rustfmt, you only need clippy running in one job and we recommend running it on Travis rather than Appveyor because Travis supports your jobs running in parallel.

We'll be adding the following to your .travis.yml:

matrix:
  include:
  - env: CLIPPY
    rust: nightly-2018-07-17
    install:
      - rustup component add clippy-preview
    script:
      - cargo clippy --all-features -- -D clippy

Highlights

  • matrix: include: is allowing us to define a complete one-off build job.
    • This will run in parallel to your tests, giving you quicker feedback.
    • No other job output will be in here, making it easier to see the results.
  • Clippy changes don't break the CI due to pinning the nightly version

Coverage

Managing Releases

The goal of this section is to help you in releasinag early and releasing often.

CHANGELOG

Providing a meaningful changelog is helpful for your users, particularly for recording breaking changes and how people can migrate between versions.

Writing a Changelog with clog

clog can help you write your changelog.

Pre-requisites:

  • Commits in conventional style.
    • gitcop is a bot to enforce a specific commit style.
  • Releases are tagged.
  • The previous release is the most recent tag.

Steps

  1. Install clog: cargo install clog-cli
  2. Run clog --setversion <X>.<Y>.<Z>
  3. Massage the output as needed.

Publishing Crates

Prebuilt Binaries

Pre-built binaries are a basic step you can take to help users of your application

  • They are easy to create
  • cargo (with install) is not meant as a binary distribution channel and has deficiencies
    • Cargo.lock is not respected
  • Users wanting to use your tool in their CI will see major slow downs and require workarounds.

Uploading Binaries to Github Releases

Pre-requisites:

  • Testing is setup.
  • Jobs exist for each supported target.

API Documentation

Wrangling Dependencies

With Rust's focos many small crates, you can end up with a plethora of dependenices. This section is to help you keep them under control.

Keeping Up on Dependencies

Challenges with dependencies:

  • Knowing when new versions are available.
  • Evaluating how new versions impact your users.
  • Validating your crate, including maintaining your oldest supported Rust version.

Thankfully this is all automatable and has been, thanks to Dependabot.

Dependabot

Recommended setup:

  1. Verify your CI configuration
  2. Sign up
  3. Add your repos
  4. Lower update frequency to once a week (to balance updates with CI load)

Your process will look like:

  1. Get a PR for a Cargo.toml or Cargo.lock update
  2. Review the release notes, changelog, and/or commit history for impact
  3. Wait until your CI gives the green light
  4. Merge

If an update introduces a conflict, Dependabot will automatically recreate the update.

Verify your CI Configuration

  • Oldest-supported rustc is used to catch dependencies that require newer rustc's
  • Don't run your CI on Dependabot branches to avoid double-running them

Limiting Branches

A snippet for .travis.yml:

branches:
  only:
  # Release tags
  - /^v\d+\.\d+\.\d+.*$/
  - master

A snippet for appveyor.yml:

branches:
  only:
  # Release tags
  - /^v\d+\.\d+\.\d+.*$/
  - master