Monthly Archives: August 2017

HowTo: Debug core dumps with ptxdist 2017.07.0

Debugging for embedded projects is a little harder than for your own computer. In many cases you can not run gdb on the target, and even if you can use gdbserver1 this does not cover all use cases. For post mortem analysis (e.g. after a segmentation fault of your program) you want to examine so called core dumps. Given you successfully found out how to let your target create those2, copied it to your workstation and unpacked it, you still need to know how to analyze it.

With the release of ptxdist 2017.07.0 the handling of debug information changed. Quoting from the release announcement:

The debug symbol handling was reworked. The debug files are now named based
on build-ids and (optional) debug IPKGs can be created. They are not
installed by default but can be installed manually as needed. This is
useful to gdb on the target or with valgrind and perf.

In my BSP those debug info is put to /usr/lib/debug in the root folder from which the target files are copied. This looks like this now:

% tree -a ./platform-foo/root/usr/lib/debug | head
platform-foo/root/usr/lib/debug
└── .build-id
    ├── 00
    │   └── ba20cb0e075c4dc0a792a9062b0864ced517b1.debug
    ├── 03
    │   └── 3b4fc351317376388fadb19fc63b4c8ab6c0d9.debug
    ├── 04
    │   ├── 01fe993aa2bed2155514c676d7001625732396.debug
    │   ├── 7bdbc5fd44a4444de24762c76a3313d1fda2c0.debug
    │   └── d0ecb6611e590a036cbdd5909cc5bfc9158af8.debug

You can imagine it is not possible anymore to load this manually, the debugger will have to find out by itself. Getting this to work caused me some headaches, but this it how I got it work: Create a file ‘gdb-config’ with the following content:3

set debug-file-directory /home/adahl/Work/bsp/foo/platform-foo/root/usr/lib/debug
set sysroot /home/adahl/Work/bsp/foo/platform-foo/root

Note: the order of the commands is important, it does not work the other way round!

Then load your core dump:

% ./platform-foo/selected_toolchain/arm-v5te-linux-gnueabi-gdb -x ./tmp/gdb-config -e ./platform-foo/root/usr/local/bin/yourtool -c ./tmp/cores/2017-08-16/core 

So you run the gdb from your toolchain, load the previously crafted file with gdb commands with -x, give the path to your executable with -e and finally the core dump file with -c and that’s it. You may now have a look at a backtrace and find out what caused the segfault …

Update: I was pointed to an easier possibility to invoke the right gdb with the necessary options in #ptxdist IRC channel (on freenode). The previous call would be like this:

% ptxdist gdb -e ./platform-foo/root/usr/local/bin/yourtool -c ./tmp/cores/2017-08-16/core 

No need to pick the correct gdb and find and configure the right directories, just add the path to your tool and your core file, ptxdist handles everything else. I bet this would also work with older ptxdist versions, where the debug symbols where placed somewhere else, but didn’t try it. This however was also just added with ptxdist 2017.07.0:

There is also a new ptxdist command ‘gdb’ for remote debugging that sets up
the sysroot correctly and a wrapper script that can be used by graphical
development environments.

  1. we already had this topic here: KDevelop: Debuggen von Programmen, die root-Rechte benötigen (German) []
  2. this would be content for another post []
  3. of course you adapt the paths to the ones you use on your machine []