pk.org: CS 419/Lecture Notes

Containment and Application Isolation

Terms and concepts you should know

Paul Krzyzanowski – 2025-10-26

General Concepts

Containment
Limits what a compromised process can do by creating isolation boundaries that confine its impact.
Isolation boundary
A security barrier that separates and restricts processes from affecting each other or the host system.
Principle of least privilege
Security principle that grants processes only the minimum permissions needed to perform their functions.

Application Sandboxing

Sandbox
A restricted execution environment that mediates interactions between an application and the operating system.
chroot
System call that changes a process's view of the root directory to a specified path.
chroot jail
An environment created by chroot where processes can only access files within a designated directory tree.
Filesystem namespace
The portion of the system that defines which files and directories a process can see.
System call interface
The mechanism through which processes request services from the kernel.
System call sandbox
Security mechanism that intercepts and filters system calls before allowing them to execute.
ptrace
Debugging interface used to monitor and control process execution, including intercepting system calls.
Time-of-check-time-of-use (TOCTOU)
Race condition vulnerability where a program changes a resource between when it's checked and when it's used.
seccomp
Secure Computing Mode that restricts which system calls a Linux process can make.
seccomp-BPF
Modern version of seccomp that uses programmable filters to control system call access.
Process virtual machine
Managed execution environment that runs code as bytecode with language-level isolation.

OS-Level Isolation Primitives

Namespace
Kernel feature that gives a process its own private view of a specific system resource.
PID namespace
Isolates process IDs so each namespace has its own set of process identifiers.
Mount namespace
Allows each namespace to have independent filesystem mount points.
Network namespace
Provides isolated network stacks with private interfaces, routing tables, and sockets.
User namespace
Maps internal user and group IDs to different values on the host system.
Control group (cgroup)
Kernel mechanism that manages and limits resource usage for groups of processes.
Controller
Component of cgroups that measures and enforces limits on a specific resource type.
Out-of-memory (OOM) handler
Kernel mechanism that terminates processes when memory limits are exceeded.
Capability
Fine-grained privilege that authorizes a specific class of privileged operations.
Root user (UID 0)
Traditional Unix superuser account with administrative privileges.

Containerization

Container
Lightweight isolated unit that packages an application with its dependencies using OS-level virtualization.
Container runtime
Software that manages containers by configuring namespaces, cgroups, and capabilities.
Image
Prebuilt snapshot containing all files, libraries, and configuration needed to run a container.
Registry
Storage system for container images that allows distribution across different hosts.
Copy-on-write filesystem
Storage system where shared data is only duplicated when modified, enabling efficient image layering.
Minimal-privilege defaults
Security configuration that starts processes with the minimum capabilities needed to function.

Virtualization

Virtual machine (VM)
Emulated computer system that runs its own operating system and kernel independently of the host.
Guest operating system
Operating system running inside a virtual machine.
Hypervisor
Software layer that sits between hardware and guest operating systems, managing their execution.
Virtual Machine Monitor (VMM)
Another term for hypervisor, emphasizing its role in monitoring and controlling virtual machines.
Trap
Mechanism where the hypervisor intercepts privileged instructions from a guest operating system.
Type 1 hypervisor
Bare-metal hypervisor that runs directly on hardware without an underlying operating system.
Type 2 hypervisor
Hosted hypervisor that runs as an application on top of a conventional operating system.
VM escape
Security vulnerability where a compromised guest gains control over the hypervisor or host system.
Hardware-level checks
Security enforcement performed by the hypervisor using hardware virtualization support.
Layered approach
Combining multiple isolation techniques (e.g., containers inside VMs) for both efficiency and security.