keywords: Windows Debugging Tools, Whitout PDB file, Crash, Dump file, Linux Core Dump, GDB debug

WinDbg (On Windows)

Motivation

PDB containes tons of informations on debugging (such as source line, symbol names), in order to prevent application from being cracked easily, we usually don’t keep PDB files while publishing application. But it would cause that function name turned into memory address in callstack of crash, thus we can’t address to source line while application crashed.
So, is there any way to address source line with PDB files removed?

Solution
  1. Get the crash callstack. e.g.:

     TestTP2   0x00000000fff20000 + 2c44aa2 
     TestTP2   0x00000000fff20000 + 987262  
     TestTP2   0x00000000fff20000 + 3ab0cc4 
     TestTP2   0x00000000fff20000 + 362fbbc 
     TestTP2   0x00000000fff20000 + 3643325 
     TestTP2   0x00000000fff20000 + 362d8e4 
     TestTP2   0x00000000fff20000 + 3642128 
     TestTP2   0x00000000fff20000 + 2c20100 
     TestTP2   0x00000000fff20000 + 39a2c25 
     TestTP2   0x00000000fff20000 + 39ba846 
     TestTP2   0x00000000fff20000 + 9cae56  
     TestTP2   0x00000000fff20000 + 9cb376  
     TestTP2   0x00000000fff20000 + 3a07c61 
     TestTP2   0x00000000fff20000 + 3a16ef8 
     TestTP2   0x00000000fff20000 + 32dff6f 
     TestTP2   0x00000000fff20000 + 32e8e3f 
     TestTP2   0x00000000fff20000 + 3146e59 
     TestTP2   0x00000000fff20000 + 43f1c9  
     TestTP2   0x00000000fff20000 + 44ec1c  
     TestTP2   0x00000000fff20000 + 44ec7a  
     TestTP2   0x00000000fff20000 + 45d265  
     TestTP2   0x00000000fff20000 + 47dc1c6 
     KERNEL32  0x0000000005b70000 + 18102   
     ntdll     0x00000000070f0000 + 5c5b4   
    
  2. Open WinDbg, set symbol file and open binary file. e.g.:
    set symbol file directory

    then WinDbg would search symbol file in this directory:

  3. Open Executable file:

  4. Execute command to address source line. e.g.

     ln TestTP2.exe+2c44aa2
    

then would output the source line. In there, because SetActorLocation is the source of engine, and we didn’t set the symbol file directory of engine binary, so it didn’t display the engine source line.
then we search the last address, we can find ourself source, and display the source file path and line
5. Finally we can locate place where application crashed at: the line above the display line

Note: the line displayed in WinDbg isn’t the exact place where cause a crash, the real line cause the crash is above it.

WinDbg download
https://github.com/dawnarc/DevTools/tree/master/Debug/Windows/WinDbg

WinDbg analysis dump file
https://www.jianshu.com/p/ee979eaadf34

Where to put the PDBs in WinDbg
https://stackoverflow.com/a/573796/1645289

Crashed without dump file (Crash on startup)

If application crashed without dump generation on progression of development, we can address source line using Visual Studio.

Open your cpp project using Visual Studio, click menu Debug -> MyGame Debug Properties, switch Configuration to Debug:

and modify Configuration Properties -> Debugging, set Command as the path of executable file of your application.

Default:

Modified:

Then startup debugging (F5), then you would get callstack with source line when you application crashed.

If your project is based on Unreal Engine, you should clean Command Arguments, or set it as an unmeaning string, (default is: "$(SolutionDir)Rouge.uproject" -skipcompile) otherwise you will get error on startup: ICU data directory was not discovered.

Windows Driver

Manually Walking a Stack
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/manually-walking-a-stack

GDB command for Linux

addr2line

Command:

addr2line -e /path/to/non-stripped/.../my-buggy-app \
    0x4a6889 0x4a8b43 0x4e8765

Or

info line *0x10045740

Using gdb to convert addresses to lines
https://stackoverflow.com/questions/8545931/using-gdb-to-convert-addresses-to-lines

Core Dump (Linux)

getting stacktrace from core dump
https://stackoverflow.com/a/5745312/1645289

How to view core files for debugging purposes in Linux
https://unix.stackexchange.com/a/202443/446854

How to get a core dump for a segfault on Linux
https://jvns.ca/blog/2018/04/28/debugging-a-segfault-on-linux/

gdb Debugging Full Example (Tutorial): ncurses
https://www.brendangregg.com/blog/2016-08-09/gdb-example-ncurses.html


人生中有些事是不得不做的,于不得不做中勉强去做,是毁灭;于不得不做中做的好,是勇敢。——叶弥《成长如蜕》