Debugging: Core Dump / Memory Dump

Creating a Core File

A core dump is essentially a snapshot of a program's memory at the exact moment of its crash. It contains the process address space, detailed in the mm_struct structure, which includes all virtual memory areas, along with any supporting information available at the time of the crash.

A core dump is generated when an operating system halts a process because of an error in the program. Often, this is due to accessing an invalid pointer, like dereferencing a NULL pointer, which causes a SEGV signal to be triggered before exiting. The operating system then tries to preserve this data in a file for later analysis.

To diagnose and debug programs, one can load the core file into a debugger, along with the executable file, to access symbols and other debugging information. Core dumps can be quite large; therefore, their maximum size can be configured using the command ulimit -c.

Consider this example code:

/* t.c */
#include <stdio.h>
void foo()
{
    int *ptr = NULL;
    *ptr = 7;
}
int main()
{
    foo();
    return 0;
}

Executing this code results in:

Segmentation fault (core dumped)

However, you may not find it in your current directory. To locate it, navigate to /proc/sys/kernel:

$ ls -la core*
-rw-r--r--. 1 root root 0 Aug 28 23:53 core_pattern
-rw-r--r--. 1 root root 0 Aug 28 16:12 core_pipe_limit
-rw-r--r--. 1 root root 0 Aug 28 23:53 core_uses_pid

Inspecting core_pattern reveals how cores are handled:

$ cat core_pattern 
|/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e

The initial “|” indicates that what follows is treated as a command by the kernel; hence, instead of writing to a file directly, it writes to that program’s standard input.

In Fedora systems like mine, cores are sent automatically to ABRT (Automatic Bug Reporting Tool). To change this behavior so that cores are saved with names like “core.[executable].[pid]”, use:

$ sudo bash -c 'echo core.%e.%p > /proc/sys/kernel/core_pattern'
$ cat /proc/sys/kernel/core_pattern
core.%e.%p

In this setting, %e is for the executable file name and %p is for the process ID.

After setting the size (in 512-byte blocks) of the core file, run the code again:

$ ulimit -c unlimited
$ ./t
Segmentation fault (core dumped)
$ ls
core.t.3209 t t.c    

Core Dump File Format

The core file was once a straightforward binary file. However, in contemporary operating systems, a process's address space might be non-sequential, and pages may be shared between processes. Consequently, the core file must encapsulate additional information and the program's state at the moment of the dump. Linux systems utilize the ELF (Executable and Linkable Format) for this purpose.

Analyzing Core File

You can use the core file with gdb:

gdb executable core-file

In our case:

$ gdb ./t core.t.3529
GNU gdb (GDB) Fedora (7.5.0.20120926-25.fc18)
...
Reading symbols from /home/khong/TEST/DMP/t...done.
[New LWP 3529]
Core was generated by `./t'.
Program terminated with signal 11, Segmentation fault.
#0  0x00000000004004fc in foo () at t.c:6
6	    *ptr = 7;

This indicates that line 6 has an issue. We know we’re dereferencing the NULL pointer at this line.

You can use backtrace to list the call stacks that were made when the program crashed:

(gdb) backtrace
#0  0x00000000004004fc in foo () at t.c:6
#1  0x0000000000400512 in main () at t.c:11

To move up and down the call stacks:

(gdb) up
#1  0x0000000000400512 in main () at t.c:11
11	    foo();
(gdb) down
#0  0x00000000004004fc in foo () at t.c:6
6	    *ptr = 7;

---

The mm_struct structure within the Linux kernel encapsulates details about a process's memory address space. Below is an outline of its principal elements:

struct mm_struct{
    struct vm_area_struct *mmap;           // List of memory areas
    struct rb_root mm_rb;                  // Red-black tree of VMAs
    struct vm_area_struct *mmap_cache;     // Last used VMA for faster access
    unsigned long (*get_unmapped_area)(struct file *filp, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags);
    unsigned long mmap_base;               // Base of mmap area
    unsigned long task_size;               // Size of task address space
    unsigned long highest_vm_end;          // Highest virtual address
    pgd_t *pgd;                            // Page global directory
    atomic_t mm_users;                     // Number of users (processes)
    atomic_t mm_count;                     // Reference count
    int map_count;                         // Number of VMAs
    struct rw_semaphore mmap_sem;          // Semaphore for mmap operations
    spinlock_t page_table_lock;            // Lock for page table operations
    struct list_head mmlist;               // List of all mm_structs
    unsigned long start_code, end_code;    // Start and end of code section
    unsigned long start_data, end_data;    // Start and end of data section
    unsigned long start_brk, brk;          // Start and end of heap
    unsigned long start_stack;             // Start of stack
    unsigned long arg_start, arg_end;      // Start and end of arguments
    unsigned long env_start, env_end;      // Start and end of environment
    unsigned long saved_auxv[AT_VECTOR_SIZE]; // Auxiliary vector
    struct mm_rss_stat rss_stat;           // Resident set size statistics
    struct linux_binfmt *binfmt;           // Binary format
    mm_context_t context;                  // Architecture-specific context
    unsigned long flags;                   // Flags
    struct core_state *core_state;         // Core dump state
    struct user_namespace *user_ns;        // User namespace
    struct file *exe_file;                 // Executable file
    struct cpumask cpumask;                // CPU mask
};

This structure includes various fields that manage memory areas, page tables, and other memory-related information for a process