Linux memory forensics has traditionally relied on debug symbols associated with specific kernel versions. However, these symbols are not typically installed on production systems by default. Sourcing them from external repositories can lead to persistent issues: repositories may become outdated, kernel builds can diverge, and analysts engaged in incident response often find that there are no available symbols for the precise kernel they need to analyze.
To tackle this challenge, Trail of Bits has introduced mquire, an open-source tool that enables the analysis of Linux memory dumps without the need for external debug information.
Understanding Memory Analysis with mquire
mquire utilizes two data sources that are embedded within Linux kernels. The first is BPF Type Format (BTF), a compact format designed to outline kernel data structure layouts. BTF supports eBPF’s compile-once, run-everywhere architecture and includes type definitions, field offsets, and type relationships. The second source is Kallsyms, which provides the same symbol address data found in /proc/kallsyms on a live system. By scanning the memory dump for Kallsyms data, mquire can combine this with BTF type information to identify and parse kernel data structures.
To utilize BTF, a kernel version of 4.18 or newer is required, with BTF enabled. This configuration has been standard in major Linux distributions for some time. Kallsyms support, however, necessitates a kernel version of 6.4 or newer, due to format changes in scripts/kallsyms.c.
Interactive SQL Interface for Memory Queries
Once a memory dump is loaded, mquire offers an interactive SQL interface, designed in the spirit of osquery, a tool that allows analysts to query the operating system state using SQL commands. The primary author of the tool, Alessandro Gario, expressed his excitement, stating, “This is something I’ve wanted to build ever since my first Querycon, where I discussed forensics capabilities with other osquery maintainers. The idea of bringing osquery’s intuitive, SQL-based exploration model to memory forensics has been on my mind for years, and mquire is the realization of that vision.”
With this SQL interface, analysts can run single queries from the command line or engage interactively. The interface supports relational queries across various data sources. For instance, analysts can join process information with open file handles or reconstruct complete file paths from kernel dentry objects, as well as cross-reference network connections with their respective processes.
The tool features a variety of queryable tables that include details about running processes such as PIDs, command lines, and binary paths; open files organized by process; memory mappings per process; active network connections; network interface data; loaded kernel modules; kernel ring buffer messages; system logs read from the kernel’s file cache; and kernel symbol addresses.
Recovering Files from the Page Cache
In addition to standard process and network enumeration, mquire possesses the capability to recover files from the kernel’s page cache. Utilizing the .dump command, it iterates through tasks and their open file descriptors to extract file contents from the page cache and save them to a specified output directory. Notably, files deleted from disk can still be recovered as long as they remain cached in memory. The tool also includes a .carve command, which allows users to extract raw memory content from a defined virtual address range.
Detecting Rootkits through Process Enumeration
mquire incorporates multiple task enumeration strategies, offering a method for detecting hidden processes. One strategy traverses the kernel’s task list, while another enumerates through the PID namespace. Rootkits may conceal processes by unlinking them from the task list while keeping them active. By comparing outputs from both enumeration methods, analysts can identify processes that appear in one list but not in the other.
Kernel-Level Scope and Future Developments
It is important to note that mquire only accesses kernel-level information. Since BTF does not include details about user-space data structures, user-space memory analysis is beyond its scope. Furthermore, the Kallsyms scanner relies on the data format from scripts/kallsyms.c, meaning that future kernel format changes may necessitate updates to the scanner's heuristics.
Trail of Bits has indicated plans for expanded table support, enhancements to caching for improved performance, and DMA-based external memory acquisition, which would facilitate the analysis of physical systems without needing to take a snapshot.
mquire is available for free on GitHub.
Essential Reads:
- 40 open-source tools redefining how security teams secure the stack
- Firmware scanning time, cost, and where teams run EMBA
