C Security & Hardening: Defeating Buffer Overflows, Format String Attacks & Memory Vulnerabilities

C Security & Hardening: Defeating Buffer Overflows, Format String Attacks & Memory Vulnerabilities
Table of Contents
- The Vulnerability Landscape
- Buffer Overflow: Stack Smashing in Detail
- Heap Overflow: Corrupting Allocator Metadata
- Format String Attacks: printf as an Attack Vector
- Integer Overflow: Silent Arithmetic Errors
- Use-After-Free: Dangling Pointer Attacks
- OS-Level Mitigations
- Compiler Hardening Flags
- Dynamic Analysis: ASan and Valgrind
- Secure Coding Standards: CERT C and MISRA-C
- Secure Code Checklist
- Frequently Asked Questions
- Key Takeaway
The Vulnerability Landscape
According to the MITRE CWE Top 25 and NIST vulnerability database, C-based software consistently dominates critical vulnerability counts:
| Vulnerability | CWE | % of CVEs | Exploitability |
|---|---|---|---|
| Buffer overflow (stack) | CWE-121 | 28% | Remote code execution |
| Heap overflow | CWE-122 | 14% | RCE via allocator corruption |
| Integer overflow | CWE-190 | 11% | Leads to buffer overflow |
| Format string | CWE-134 | 8% | Read/write arbitrary memory |
| Use-after-free | CWE-416 | 7% | Type confusion attacks |
| NULL dereference | CWE-476 | 12% | Denial of service |
Buffer Overflow: Stack Smashing in Detail
The classic exploit: overwrite the return address to redirect execution to attacker-controlled code:
// VULNERABLE: classic textbook buffer overflow
void vulnerable_login(void) {
char username[16]; // 16-byte stack buffer
gets(username); // NO BOUNDS CHECK — reads until newline
// Input: "AAAAAAAAAAAAAAAA" * 4 + <return_address>
// Overwrites: return address → points to attacker shellcode
printf("Welcome, %s\n", username);
}
// SECURE: bounds-limited input
void secure_login(void) {
char username[64];
if (!fgets(username, sizeof(username), stdin)) {
return; // EOF or error
}
// Remove trailing newline
username[strcspn(username, "\n")] = '\0';
// Validate length
if (strlen(username) == 0 || strlen(username) > 32) {
fprintf(stderr, "Username must be 1-32 characters\n");
return;
}
printf("Welcome, %s\n", username);
}Dangerous → Secure function replacements:
| Dangerous | Secure Alternative | Why Safer |
|---|---|---|
gets() | fgets(buf, size, fp) | Size-bounded |
strcpy(dst, src) | strncpy(dst, src, size-1) + null-terminate | Size-bounded |
strcat(dst, src) | strncat(dst, src, size - strlen(dst) - 1) | Size-bounded |
sprintf(buf, fmt, ...) | snprintf(buf, size, fmt, ...) | Size-bounded, no overflow |
scanf("%s", buf) | scanf("%63s", buf) | Width-limited in format string |
Heap Overflow: Corrupting Allocator Metadata
Heap overflows are less predictable than stack overflows but often more serious because they corrupt malloc's internal bookkeeping:
// VULNERABLE: heap overflow
int *buf = malloc(10 * sizeof(int)); // Allocate 10 integers
for (int i = 0; i <= 10; i++) { // Bug: writes 11 items (one past end)
buf[i] = i; // buf[10] overwrites glibc heap metadata
}
// After buf[10]: heap's internal "next free block" pointer is corrupted
// Next malloc/free call: undefined behavior, possible RCE via heap spray
// SECURE: strict bounds
int *buf = malloc(10 * sizeof(int));
if (!buf) return -1;
const int COUNT = 10;
for (int i = 0; i < COUNT; i++) { // Strict < not <=
buf[i] = i;
}
free(buf);Detect with AddressSanitizer:
gcc -fsanitize=address -g vulnerable.c -o vulnerable
./vulnerable
# ASan output: heap-buffer-overflow on address 0x... at pc 0x...Format String Attacks: printf as an Attack Vector
Passing user input directly as a printf format string allows reading and writing arbitrary memory:
// CRITICAL VULNERABILITY: user controls format string
void vulnerable_log(const char *user_message) {
printf(user_message); // If user inputs "%x %x %x %n":
// %x %x %x: READ from stack (information disclosure)
// %n: WRITE the count of printed chars to the next pointer on stack
// → arbitrary write to arbitrary address
}
// ATTACK INPUT: "%08x %08x %08x %n"
// Reads: stack addresses, heap addresses (information disclosure)
// Writes: can overwrite function's GOT entry → execute shellcode
// SECURE: ALWAYS use %s for strings
void secure_log(const char *user_message) {
printf("%s", user_message); // user_message treated as data, not format
fprintf(stderr, "%s\n", user_message); // Same for stderr
syslog(LOG_INFO, "%s", user_message); // Same for syslog
}Rule: Never pass untrusted data as the format argument to any printf-family function. Always use printf("%s", data) or snprintf(buf, size, "%s", data).
Integer Overflow: Silent Arithmetic Errors
Integer overflow causes incorrect results that lead to undersized buffers being allocated:
#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>
// VULNERABLE: allocation size overflow
void vulnerable_alloc(size_t width, size_t height) {
// If width = 65537, height = 65537 on 32-bit:
// width * height = 65537 * 65537 = 4,295,098,369
// Truncated to uint32_t: 65,537 (far too small!)
size_t size = width * height * sizeof(uint32_t); // OVERFLOW!
uint32_t *pixels = malloc(size); // malloc(tiny_value)
pixels[width * height - 1] = 0; // HEAP OVERFLOW
}
// SECURE: check before multiplying
int safe_alloc(size_t width, size_t height, uint32_t **out) {
// Check: would width * height overflow size_t?
if (width > 0 && height > SIZE_MAX / width) {
return -1; // Would overflow
}
size_t count = width * height;
if (count > SIZE_MAX / sizeof(uint32_t)) {
return -1; // Multiplication with sizeof would overflow
}
*out = malloc(count * sizeof(uint32_t));
return *out ? 0 : -1;
}
// C23 approach: use __builtin_mul_overflow (GCC/Clang)
bool safe_multiply(size_t a, size_t b, size_t *result) {
return !__builtin_mul_overflow(a, b, result); // true if no overflow
}Use-After-Free: Dangling Pointer Attacks
#include <stdlib.h>
#include <stdio.h>
// VULNERABLE: use-after-free
void vulnerable_pattern(void) {
int *p = malloc(sizeof(int));
*p = 42;
free(p);
// Bug: p still contains the old address (now freed)
printf("%d\n", *p); // Use-after-free — undefined behavior
*p = 99; // WRITE to freed memory
// If malloc re-allocates this block, writing 99 corrupts new object's data
}
// SECURE: null after free, RAII pattern
void secure_pattern(void) {
int *p = malloc(sizeof(int));
if (!p) return;
*p = 42;
free(p);
p = NULL; // Immediately null the pointer after free
if (p) { // This check now protects against use-after-free
printf("%d\n", *p);
}
}
// Better: wrapper macro
#define SAFE_FREE(ptr) do { free(ptr); (ptr) = NULL; } while (0)OS-Level Mitigations
Modern operating systems deploy multiple layers of exploit mitigation:
# Check what mitigations your binary has on Linux:
checksec --file=./your_binary
# Typical output:
# Stack Canary : ENABLED → Detects stack overflows before ret
# NX : ENABLED → Stack/heap marked non-executable (no shellcode)
# PIE : ENABLED → Position-independent; enables ASLR
# ASLR : ENABLED → Randomize stack/heap/library addresses
# RELRO : FULL → GOT table read-only after initStack Canaries: The compiler places a random 8-byte value (the "canary") before the return address. Before a function returns, the canary is checked. If overwritten (by a buffer overflow), the program is immediately terminated with a fault.
ASLR (Address Space Layout Randomization): Randomizes where the stack, heap, and libraries are loaded each run. Without knowing the exact address of the return address or the target shellcode, most exploits fail.
NX/DEP (No-Execute/Data Execution Prevention): Marks the stack and heap as non-executable. Even if an attacker overwrites the return address, they cannot execute code on the stack — they must use Return-Oriented Programming (ROP), which is significantly more complex.
Compiler Hardening Flags
Always compile production C with these security flags:
# Recommended production build flags
gcc -O2 -Wall -Wextra -Wpedantic \
-fstack-protector-strong \ # Stack canaries
-D_FORTIFY_SOURCE=3 \ # Runtime bound checking on libc calls
-fpie -pie \ # Enable ASLR
-Wl,-z,relro,-z,now \ # Full RELRO
-Wl,-z,noexecstack \ # NX stack
-fstack-clash-protection \ # Stack clash defense
-fcf-protection=full \ # Control flow integrity (Intel CET)
main.c -o program
# Debug/testing flags (never in production release)
gcc -g -fsanitize=address,undefined,leak \ # ASan + UBSan + LeakSanitizer
-fsanitize=thread \ # + ThreadSanitizer (not with ASan)
main.c -o program_debug
# Static analysis
cppcheck --enable=all main.c # Static analysis
clang --analyze main.c # Clang analyzer
scan-build gcc main.c # Clang scan-build-D_FORTIFY_SOURCE=3 (C23 recommended): The compiler replaces unsafe functions like strcpy, memcpy, sprintf with bounds-checking versions that abort() at runtime if an overflow is detected. Zero code change required — just a compile flag.
Dynamic Analysis: ASan and Valgrind
# AddressSanitizer (ASan) — fast, finds most memory errors
gcc -fsanitize=address,undefined -g program.c -o program_asan
./program_asan
# Catches: heap/stack buffer overflow, use-after-free, double-free,
# use-after-return, invalid free, memory leaks
# Valgrind memcheck — slower but comprehensive
gcc -g program.c -o program_debug
valgrind --tool=memcheck --leak-check=full --track-origins=yes ./program_debug
# libFuzzer — automated security testing (finds inputs that crash your code)
gcc -fsanitize=address,fuzzer program.c -o fuzz_target
./fuzz_target corpus_dir/ # Generates random inputs, tracks crashesSecure Coding Standards: CERT C and MISRA-C
CERT C Secure Coding Standard (developed by Carnegie Mellon CERT):
- Rules like
MEM30-C: Do not access freed memory. - Rules like
INT30-C: Ensure unsigned integer operations do not wrap. - Used by: US Department of Defense, automotive suppliers, avionics.
- Tool:
cppcheck --addon=cert main.c
MISRA-C 2023 (Motor Industry Software Reliability Association):
- Originally for automotive safety (ISO 26262 functional safety).
- Stricter subset of C — forbids goto, dynamic allocation, function pointers.
- Used by: automotive (BMW, Bosch), aerospace (Boeing), medical devices (FDA regulated).
- Compliance verified by: PC-lint, LDRA, Polyspace.
Secure Code Checklist
Before shipping any C library or service:
- All array accesses bounds-checked
-
snprintfused (neversprintf,strcpy,gets) -
mallocreturn values always checked for NULL -
free()followed immediately byptr = NULL - All
errno-setting functions have return value checked -
printf("%s", data)neverprintf(data) - Integer arithmetic checked for overflow before allocation sizing
-
memset_explicit()(C23) used for zeroing credentials/keys - Build with:
-fstack-protector-strong -D_FORTIFY_SOURCE=3 -pie - Tested with: AddressSanitizer on all code paths
- Tested with: Valgrind for memory leak verification
- Fuzz tested with: libFuzzer or AFL++ for all input-parsing code
Frequently Asked Questions
Is C fundamentally insecure? C is memory-unsafe by design — the language gives you raw memory access with no bounds checking. This is intentional for performance. However, with disciplined use of safe functions, bounds checking, compiler hardening flags, and static/dynamic analysis, C software can achieve excellent security. The Linux kernel, OpenSSH, and OpenSSL run on billions of systems with acceptable security records.
Why isn't FORTIFY_SOURCE on by default?
It has a measurable performance overhead (additional branch per function call) that some applications cannot afford. High-performance C code often runs with -D_FORTIFY_SOURCE=0 and relies on code review and fuzzing instead.
What is the difference between ASan and Valgrind? ASan is ~2-5× slower than native execution and is a compile-time instrumentation. Valgrind is ~10-20× slower and instruments at runtime (no recompilation needed). ASan finds more types of errors (use-after-return, stack buffer overflows) and is faster. Valgrind is more accurate for certain leak analysis and doesn't require recompilation — useful for testing without source code.
Should I prefer Rust over C for new security-sensitive code? For new code where Rust is feasible — yes. Rust's ownership model and borrow checker eliminate entire categories of memory vulnerabilities at compile time. For existing C codebases, FFI boundaries with Rust are a practical path. For systems that must remain in C (OS kernels, embedded with limited tooling), C with aggressive use of analysis tools is still the state of practice.
Key Takeaway
C security is not accidental — it is a Deliberate Engineering Practice. Every buffer using user-supplied size, every pointer cast, every format string, every integer arithmetic involving user data is a potential attack surface. By systematically applying safe functions, compiler hardening flags, and dynamic analysis tools, you transform C from "powerful but dangerous" into "powerful and professionally hardened."
Read next: Project: Heap Memory Allocator — Build malloc & free from Scratch →
Part of the C Mastery Course — 30 modules from C basics to expert systems engineering.
