Optional Lab: Rust Memory Safety

Expect it to take about 30-60 minutes.

Overview

Labs 1–3 attacked C programs: the bugs were a missing bounds check, a pointer that outlived its object, or user input that became a format string. In lecture we saw that most of these memory-corruption bugs become compile errors in Rust. The compiler tracks who owns each value and how long every reference lives, and it refuses to build a program in which a reference could outlive its data, alias a value that is being mutated, or be shared between threads unsafely.

In this lab you will trigger those errors yourself, read what the compiler is telling you, and fix the programs. Then you will write two small functions that handle untrusted input — an integer-overflow allocation size and the Heartbleed length field — without crashing, and finally make a multi-threaded counter compile and count correctly.

Topics covered:

• Ownership and moves, borrowing, and lifetimes
• How borrow-checker errors map to C bug classes (use-after-free, dangling pointers, iterator invalidation, …)
• Checked integer arithmetic and bounds-checked slicing on untrusted input
• Send/Sync, Arc, and Mutex: why data races are compile errors
• What unsafe does — and does not — turn off

Background reading

• The Rust Programming Language (“the Book”), Chapter 4: Understanding Ownership and Chapter 16: Fearless Concurrency
• rustc --explain <code> — every error code the compiler prints (e.g. E0382) comes with a long-form explanation and examples
• CVE-2014-0160 (Heartbleed), for the background to Part B

Rules for every part

• No unsafe. Every file starts with #![forbid(unsafe_code)]; leave it there.
• No .clone() in Part A. Copying the data makes the error go away without teaching you anything.
• Keep each program’s printed output exactly as shown in the comment next to its println!.
• Make the smallest change that works. A good fix is usually one or two lines.

Before you begin

Install Rust from https://rustup.rs if you do not have it, and check the version (the project uses the 2024 edition, which needs 1.85 or newer; run rustup update if yours is older):

$ rustc --version

Download the lab files and unpack them:

mkdir -p ~/cs487/rust-lab && cd ~/cs487/rust-lab
wget https://xiaoguang.wang/computer-security-labs/labs/files/rust-lab.tar.gz
# (no wget on macOS? use: curl -O <same URL>)
tar xzf rust-lab.tar.gz
cd Labsetup && ls

You should see:

Cargo.toml              # the package manifest
src/lib.rs              # Part B: two functions to implement, plus 6 unit tests
src/bin/a1_moved.rs     # Part A1
src/bin/a2_invalidation.rs
src/bin/a3_dangling.rs
src/bin/c_counter.rs    # Part C

Try building the first program:

$ cargo build --bin a1_moved

You should see a compile error. That is expected: Parts A and C start broken, and the error is the thing you are studying.

Note on cargo test. Plain cargo test compiles every program in the package, including the broken ones, so it fails until Parts A and C are fixed. While working on Part B, use cargo test --lib, which builds only src/lib.rs.

Part A: Fixing the Borrow Checker

Each of the three programs below fails to compile. Each one is the Rust version of a classic C memory bug; in C it would compile without complaint and misbehave at run time.

For each program:

1. Build it and read the whole error message, not just the first line. Rust points at two places: where the problem starts (a move, a borrow, a scope ending) and where it bites. Note the error code, e.g. E0382.
2. Make the smallest change that makes it compile and print the expected output. Respect the per-file constraints in the comments at the top (no .clone(); in a3_dangling.rs, keep longest_word unchanged).
3. Decide which C bug class the error stopped: use-after-free, double free, dangling pointer (to a stack variable), iterator invalidation, …

Hint. For A1, ask who owns words after the call to total_len, and whether total_len actually needs ownership. For A2, think about what push may do to the vector’s heap buffer, and what best points into. For A3, the reference is fine — the thing it refers to is not.

Understanding check. In A2, the equivalent C++ program (std::vector, a pointer or iterator to the max element, then push_back) often appears to work. Why does it sometimes work and sometimes print garbage, and why is “sometimes works” the dangerous case for security?

Part B: Handling Untrusted Input

Memory safety is not only about pointers. Two of the most famous C vulnerabilities came from trusting a number the attacker controlled:

• Integer overflow in an allocation size. malloc(count * elem_size + HEADER) wraps around silently when the product is too large, so the program allocates a tiny buffer and then writes count elements into it — a heap overflow.
• Heartbleed (CVE-2014-0160). An OpenSSL heartbeat message carried a length field and a payload. The server copied length bytes back to the client without checking that the payload was really that long, leaking up to 64 KB of server memory — private keys included — per request.

Open src/lib.rs and replace the two todo!()s:

• alloc_size(count, elem_size) returns Some(count * elem_size + HEADER), or None if any step overflows.
• parse_payload(packet) treats the first byte as a length n and returns the n-byte payload that follows it, or None if the packet is empty or holds fewer than n bytes after the length byte. Bytes after the payload are ignored.

Neither function may panic on any input — a panic on attacker input is a denial-of-service bug, even if it is not memory corruption. Check your work with:

$ cargo test --lib

All 6 tests must pass.

Hint. Look at checked_mul and checked_add on integers, and split_first and get on slices. The ? operator works on Option too: x? returns None from the function early if x is None. Each function fits in one or two lines.

Part C: Sharing State Between Threads

src/bin/c_counter.rs starts 8 threads that each increment one shared counter 1000 times. It uses Rc<RefCell<i32>> — the usual single-threaded way to share a mutable value — and it does not compile.

$ cargo run --bin c_counter

Change the types so that it compiles and always prints 8000. Keep the 8 threads and the loop structure; you will need to change the use lines and a few method calls to match the new types.

Hint. Look for the thread-safe counterpart of Rc and the thread-safe counterpart of RefCell in std::sync. Read the error message carefully: it names the trait that is missing and the type that lacks it.

Run the fixed program several times to convince yourself the answer is always 8000.

Final check

Before writing up, confirm that everything builds and passes together:

$ cargo build --bins && cargo test

Both commands must finish with no errors. Include this output in your report.

Cleaning up

cargo clean deletes the target/ build directory (a few hundred MB). You can delete the whole Labsetup/ folder once your report is submitted.

Submission

Submit a single PDF report. Do not upload your source files, a zip of the project, or a separate answers file — the report is the only thing that is graded, so everything must be in it:

CS487 Optional Lab: Rust Memory Safety — <Your Name>, <Your NetID>
  1. Environment
     - OS, and the output of `rustc --version`
  2. Part A: Fixing the Borrow Checker
     - Summary table, then A1, A2, A3: error, fix, output, explanation
  3. Part B: Handling Untrusted Input
     - Function bodies, test output, answers
  4. Part C: Sharing State Between Threads
     - Error, fix, output, answers
  5. Final check
     - Output of `cargo build --bins && cargo test`

Add screenshots of important code and execution outputs. Show only the lines you changed, not whole files. As in every lab, a fix with no explanation will not receive credit — the explanation of what the C version would have done is the point of the exercise.

See Submitting a Lab Report for general formatting guidance.