Differences

This shows you the differences between two versions of the page.

--- en:odroid_linux_kernel_debugging [2015/09/24 14:50]
odroid [Physical connection]
+++ en:odroid_linux_kernel_debugging [2015/09/24 14:51] (current)
odroid [Physical connection]
@@ Line 1: / Line 1: @@
+===== How to use the KGDB =====
+KGDB is a source level debugger for the kernel from host machine to the target board. It allows kernel developers to use arch specific gdb to debug the kernel in a similar way to debugging application programs. KGDB patch already included in the kernel tree.
+KDGB support for ARCH like x64, arm, arm64, mips.
+KGDB debugging is quit similar to user-space debugging, but it's more complex when it come to SMP arch. Kernel bug are pretty difficult to understand until and unless you understand the call trace that is generated.
+<WRAP center round important 100%>
+Internal debug information of the ARCH is need in order to debug kernel programming.
+</WRAP>
+==== Physical connection ====
+KDGB debug can be done using serial port (COM1), USB or Ethernet.\\
+Well support for serial interface depends on the serial driver supporting CONFIG_CONSOLE_POLL features.
+=== Serial interface ===
+The UART port of the 30-pin header on ODROID-XU4 is accessible via /dev/ttySAC0/\\
+The UART port can be connected to your host PC via [[http://www.hardkernel.com/main/products/prdt_info.php?g_code=G134111883934|USB-UART Kit]]
+Here is a quick-and-dirty diagram for your DIY debugging tool.\\
+{{http://dn.odroid.com/wiki_image/kgdb_connection_3.png|}}\\
+We connected four wires on a 30-pin header socket and add the hot-melt-glue on that.\\
+{{http://dn.odroid.com/wiki_image/kgdb_connection_1.jpg|}}\\
+And plug the socket into the XU4.\\
+{{http://dn.odroid.com/wiki_image/kgdb_connection_2.jpg|}}\\
+==== Configure the kernel to enable the KGDB ====
+In order to configure KGDB you need to setup development machine to compile the kernel.
+First step is to setup the toolchain.
+Note: You need to use toolchain specific gdb to communicate with the target board. i.e **arm-linux-gnueabihf-gdb**. It better to use the same toolchain debugger to debug in-order to interpreter the assembly code correctly
+Toolchain (4.9.2 Click one of the site to download toolchain to build Linux kernel).
+  * [[http://releases.linaro.org/14.09/components/toolchain/binaries/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux.tar.xz|Download #1]]
+  * [[http://dn.odroid.com/toolchains/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux.tar.xz|Download #2]]
+Once the download is done, extract the tarball to **/opt/toolchains/**.
+<code>
+$ sudo mkdir -p /opt/toolchains
+$ sudo tar xvf gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux.tar.xz -C /opt/toolchains/
+</code>
+In order to add the toolchain path to PATH, paste below lines to **$HOME/.bashrc**.
+<code>
+export ARCH=arm
+export CROSS_COMPILE=arm-linux-gnueabihf-
+export PATH=/opt/toolchains/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/:$PATH
+</code>
+You can check if the toolchain installed above works properly while checking the version of toolchain. If you can find **gcc version 4.9.2 20140904 (prerelease)** at the end of the line, the toolchain is well installed.
+<code>
+$ arm-linux-gnueabihf-gcc -v
+Using built-in specs.
+COLLECT_GCC=/opt/toolchains/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/arm-linux-gnueabihf-gcc
+COLLECT_LTO_WRAPPER=/opt/toolchains/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/../libexec/gcc/arm-linux-gnueabihf/4.9.2/lto-wrapper
+Target: arm-linux-gnueabihf
+Configured with: /cbuild/slaves/oorts/crosstool-ng/builds/arm-linux-gnueabihf-linux/.build/src/gcc-linaro-4.9-2014.09/configure --build=i686-build_pc-linux-gnu --host=i686-build_pc-linux-gnu --target=arm-linux-gnueabihf --prefix=/cbuild/slaves/oorts/crosstool-ng/builds/arm-linux-gnueabihf-linux/install --with-sysroot=/cbuild/slaves/oorts/crosstool-ng/builds/arm-linux-gnueabihf-linux/install/arm-linux-gnueabihf/libc --enable-languages=c,c++,fortran --disable-multilib --enable-multiarch --with-arch=armv7-a --with-tune=cortex-a9 --with-fpu=vfpv3-d16 --with-float=hard --with-pkgversion='crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09' --with-bugurl=https://bugs.launchpad.net/gcc-linaro --enable-__cxa_atexit --enable-libmudflap --enable-libgomp --enable-libssp --with-gmp=/cbuild/slaves/oorts/crosstool-ng/builds/arm-linux-gnueabihf-linux/.build/arm-linux-gnueabihf/build/static --with-mpfr=/cbuild/slaves/oorts/crosstool-ng/builds/arm-linux-gnueabihf-linux/.build/arm-linux-gnueabihf/build/static --with-mpc=/cbuild/slaves/oorts/crosstool-ng/builds/arm-linux-gnueabihf-linux/.build/arm-linux-gnueabihf/build/static --with-isl=/cbuild/slaves/oorts/crosstool-ng/builds/arm-linux-gnueabihf-linux/.build/arm-linux-gnueabihf/build/static --with-cloog=/cbuild/slaves/oorts/crosstool-ng/builds/arm-linux-gnueabihf-linux/.build/arm-linux-gnueabihf/build/static --with-libelf=/cbuild/slaves/oorts/crosstool-ng/builds/arm-linux-gnueabihf-linux/.build/arm-linux-gnueabihf/build/static --enable-threads=posix --disable-libstdcxx-pch --enable-linker-build-id --enable-plugin --enable-gold --with-local-prefix=/cbuild/slaves/oorts/crosstool-ng/builds/arm-linux-gnueabihf-linux/install/arm-linux-gnueabihf/libc --enable-c99 --enable-long-long --with-mode=thumb --disable-multilib --with-float=hard
+Thread model: posix
+gcc version 4.9.2 20140904 (prerelease) (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09)
+$ arm-linux-gnueabihf-gdb -v
+GNU gdb (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09) 7.6.1-2013.10
+Copyright (C) 2013 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
+and "show warranty" for details.
+This GDB was configured as "--host=i686-build_pc-linux-gnu --target=arm-linux-gnueabihf".
+For bug reporting instructions, please see:
+<https://bugs.launchpad.net/gcc-linaro>.
+</code>
+==== Enable the KGDB support int the Kernel.====
+Enable KDGB and debug flags.
+| CONFIG_KALLSYMS | Load all symbols for debugging/ksymoops |
+| CONFIG_KALLSYMS_ALL | Load all symbols for debugging/ksymoops |
+| CONFIG_DEBUG_INFO | kernel to be built with full debugging information included |
+| CONFIG_MAGIC_SYSRQ | Enable magic SysRq key functions by default, uses to print the current status of stack trace. |
+| CONFIG_FRAME_POINTER | This option inserts code to into the compiled executable which saves the frame information in registers or on the stack at different points which allows a debugger such as gdb to more accurately construct stack back traces while debugging the kernel. |
+| CONFIG_KGDB | Enable kernel debugger, it depends on the ARCH supported. |
+| CONFIG_KGDB_SERIAL_CONSOLE | Use kgdb over the serial console, Serail driver should support CONSOLE_POLL feature in order to support KGDB |
+<code>
+ Kernel hacking --->
+    [*] Kernel debugging
+    [*] KGDB: kernel debugging with remote gdb  --->
+      --- KGDB: kernel debugging with remote gdb
+      <*>   KGDB: use kgdb over the serial console
+</code>
+Once All the config flags are Enables. You need to cross compile the kernel and put the image on the kernel.
+==== How to initiate the KGDB connection ====
+** Host Side Configurations **
+Using UART connection from the target board to the host Linux machine (x86/x64) we need to setup 'minicom' or 'picocom'
+Note : You need to identify the port connection from the dmesg.
+**$ sudo picocom -b 115200 -r -l /dev/ttyUSB0**
+Once you verify the connection is correct and you are able to see the boot messages. You can setup KGDB connection with the target board.
+** Target Board Configuration **
+On the target board we need to initialize KGDB by setting up the kernel command line with **kgdbwait kgdboc=ttySAC2,115200**
+You need to update the kernel command line for the bootloader u-boot in the boot.ini
+<code>setenv bootrootfs "console=tty1 kgdbwait kgdboc=ttySAC2,115200 console=ttySAC2,115200n8 root=UUID=e139ce78-9841-40fe-8823-96a304a09859 rootwait ro fsck.repair=yes" </code>
+Note:__ It's not mandatory to set the KGDB option in the kernel command line. __
+** kgdbwait **
+makes kgdb wait for a debugger connection during booting of a kernel. You can only use this option you compiled a kgdb I/O driver into the kernel and you specified the I/O driver configuration as a kernel command line option.
+The kernel will stop and wait as early as the I/O driver and architecture allows when you use this option.
+** kgdboc **
+kgdboc driver was originally an abbreviation meant to stand for "kgdb over console". Kgdboc is designed to work with a single serial port. It was meant to cover the circumstance where you wanted to use a serial console as your primary console as well as using it to perform kernel debugging
+kgdboc=<tty-device>,[baud]
+Target is loaded with new kernel and KGDB is enabled in command prompt.
+Note: Binding the kdb/kgdb feature to a serial port, by writing a value into the sys filesystem. This step is run time.
+**Target Board**
+<code>
+$ echo ttySAC2 > /sys/module/kgdboc/parameters/kgdboc
+$ echo g > /proc/sysrq-trigger
+</code>
+**Host side**
+<code>
+$arm-none-linux-gnueabi-gdb vmlinux
+gdb > set remotebaud 115200
+gdb > set debug remote 1
+gdb > target remote /dev/ttyUSB0
+</code>
+===== Verification of KDGB connection ======
+**Target Board**
+Once you power on the target board, After initial boot of CPU it will break the execution of booting.
+<code>
+[    0.972718] [c7] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
+[    0.975561] [c7] 12c00000.serial: ttySAC0 at MMIO 0x12c00000 (irq = 83) is a S3C6400/10
+[    0.976365] [c7] samsung-pinctrl 14010000.pinctrl: pin gpa0-7 already requested by 12c80000.i2c; cannot claim for 12c10000.serial
+[    0.976423] [c7] samsung-pinctrl 14010000.pinctrl: pin-178 (12c10000.serial) status -22
+[    0.976468] [c7] samsung-pinctrl 14010000.pinctrl: could not request pin 178 on device samsung-pinctrl
+[    0.976515] [c7] samsung-uart 12c10000.serial: Error applying setting, reverse things back
+[    0.976673] [c7] 12c10000.serial: ttySAC1 at MMIO 0x12c10000 (irq = 84) is a S3C6400/10
+[    0.977459] [c7] 12c20000.serial: ttySAC2 at MMIO 0x12c20000 (irq = 85) is a S3C6400/10
+[    2.578493] [c7] console [ttySAC2] enabled
+[    2.583257] [c7] 12c30000.serial: ttySAC3 at MMIO 0x12c30000 (irq = 86) is a S3C6400/10
+[    2.592651] [c7] kgdb: Registered I/O driver kgdboc.
+[    2.596170] [c7] kgdb: Waiting for connection from remote gdb...
+Entering kdb (current=0xdf118000, pid 1) on processor 7 due to Keyboard Entry
+[7]kdb>
+</code>
+You can give command for help or console for verification.
+If you want to break and boot the board you need to specify the '**go**'.
+In order to connect to gdb via kgdboc, the kernel must first be stopped. There are several ways to stop the kernel which include using kgdbwait as a boot argument, via a sysrq-g, or running the kernel until it takes an exception where it waits for the debugger to attach.
+**$ echo g > /proc/sysrq-trigger**
+<code>
+root@odroidxu4:/usr/src/crypto_dev# echo g > /proc/sysrq-trigger
+[  135.581559] [c2] SysRq : DEBUG
+Entering kdb (current=0xdcdf6b80, pid 1012) on processor 2 due to Keyboard Entry
+[2]kdb>
+[2]kdb>
+[2]kdb> help
+Command         Usage                Description
+----------------------------------------------------------
+md              <vaddr>             Display Memory Contents, also mdWcN, e.g. md8c1
+mdr             <vaddr> <bytes>     Display Raw Memory
+mdp             <paddr> <bytes>     Display Physical Memory
+mds             <vaddr>             Display Memory Symbolically
+mm              <vaddr> <contents>  Modify Memory Contents
+go              [<vaddr>]           Continue Execution
+rd                                  Display Registers
+rm              <reg> <contents>    Modify Registers
+ef              <vaddr>             Display exception frame
+bt              [<vaddr>]           Stack traceback
+btp             <pid>               Display stack for process <pid>
+bta             [D|R|S|T|C|Z|E|U|I|M|A]
+                                    Backtrace all processes matching state flag
+btc                                 Backtrace current process on each cpu
+btt             <vaddr>             Backtrace process given its struct task address
+env                                 Show environment variables
+set                                 Set environment variables
+help                                Display Help Message
+?                                   Display Help Message
+cpu             <cpunum>            Switch to new cpu
+</code>
+**Host Machine**
+Kernel source code is present on the host machine and we can analyze the execution of the target board using gdb.
+All the debug symbols are present into the binary vmlinux and kernel modules. Taking the reference of the source code we need to load the symbols of the module to watch.
+Note: Kernel have lots of functionality any is call asynchronously. So putting an breakpoints its bit tedious to debug the bug.
+<code>
+$ arm-linux-gnueabihf-gdb ./vmlinux
+GNU gdb (Linaro GDB 2014.11) 7.8-2014.09-1-git
+Copyright (C) 2014 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
+and "show warranty" for details.
+This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=arm-linux-g                                                                              nueabihf".
+Type "show configuration" for configuration details.
+For bug reporting instructions, please see:
+<https://bugs.launchpad.net/gcc-linaro>.
+Find the GDB manual and other documentation resources online at:
+<http://www.gnu.org/software/gdb/documentation/>.
+For help, type "help".
+Type "apropos word" to search for commands related to "word"...
+Reading symbols from ./vmlinux...done.
+(gdb) target remote /dev/ttyUSB0
+Remote debugging using /dev/ttyUSB0
+queue_stop_cpus_work (cpumask=0x0 <__vectors_start>, fn=0x1 <__vectors_start>,
+    arg=0xc0d6aba4 <futex_queues+8584>, done=0xc0d6aba0 <futex_queues+8580>)
+    at kernel/stop_machine.c:145
+     {
+(gdb)
+(gdb)
+(gdb) list
+     static DEFINE_PER_CPU(struct cpu_stop_work, stop_cpus_work);
+     static void queue_stop_cpus_work(const struct cpumask *cpumask,
+                                      cpu_stop_fn_t fn, void *arg,
+                                      struct cpu_stop_done *done)
+     {
+             struct cpu_stop_work *work;
+             unsigned int cpu;
+             /* initialize works and done */
+(gdb)
+             for_each_cpu(cpu, cpumask) {
+</code>
+==== A simple example to trace a tiny device driver ====
+**Target Boards : OOPS.**
+Kernel module for debug source code.
+<code>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+static void create_oops() {
+        *(int *)0 = 0;
+}
+static int __init my_oops_init(void) {
+        printk("oops from the module\n");
+        create_oops();
+       return (0);
+}
+static void __exit my_oops_exit(void) {
+        printk("Goodbye world\n");
+}
+module_init(my_oops_init);
+module_exit(my_oops_exit);
+</code>
+Note: As this is static code once loaded  in memory it break into kgbd session
+<code>
+root@odroidxu4d:~# echo ttySAC2 > /sys/module/kgdboc/parameters/kgdboc
+root@odroidxu4d:~#
+root@odroidxu4d:~#
+root@odroidxu4d:~#
+root@odroidxu4d:~# insmod /home/odroid/Downloads/debug/code.ko
+[   81.724023] [c4] Unable to handle kernel NULL pointer dereference at virtual                                                                               address 00000000
+[   81.731617] [c4] pgd = dc6f4000
+[   81.734262] [c4] [00000000] *pgd=00000000
+[   81.738406] [c4] Internal error: Oops: 805 [#1] PREEMPT SMP ARM
+Entering kdb (current=0xde242b00, pid 1178) on processor 4 Oops: (null)
+due to oops @ 0xbf11501c
+dCPU: 4 PID: 1178 Comm: insmod Tainted: P           O 3.10.82-xu4hk #3
+dtask: de242b00 ti: dad30000 task.ti: dad30000
+PC is at my_oops_init+0x1c/0x24 [code]
+LR is at vprintk_emit+0x1b4/0x660
+pc : [<bf11501c>]    lr : [<c002de74>]    psr: 40080013
+sp : dad31e30  ip : dad319b8  fp : dad31e3c
+r10: 00000000  r9 : dca463f0  r8 : dca463c0
+r7 : 00000001  r6 : bf113068  r5 : bf115000  r4 : c09aaec0
+r3 : 60080013  r2 : de242b00  r1 : 00000000  r0 : 00000000
+Flags: nZcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
+Control: 10c5387d  Table: 5c6f406a  DAC: 00000015
+CPU 0 / CLUSTER 0
+DFSR: 00000805, ADFSR: 00000000, IFSR: 00000007, AIFSR: 00000000
+CPUMERRSR: 00000000_00000000, L2MERRSR: 00000000_00000000
+LR: 0xc002ddf4:
+ddf4  e58d200c e51b2418 e58d2008 e1a0200c e1a0100a e51b0410 e58d3014 e1cd80f0
+de14  ebfff987 ebfffd7c e51b341c e3500000 e0855003 0a000050 e30e213c e3570000
+de34  e34c2064 e287301f e5922000 a1a03007 e1a032c3 e207701f e7923103 e1a07733
+de54  e3170001 0a000095 e3e03000 e59f049c e5843104 eb185acc ebfffe85 eb017ef7
+de74  e51b3414 e3130080 051b3414 e121f003 e1a00005 e24bd028 e89daff0 e5d6292c
+de94  e30b3f50 e34c309a e3520001 1affffc1 e5d3392d e3530000 0affffbe e3530030
+deb4  3affffbc e3530037 9a0000ef e3530064 1affffb8 e59f943c e3e02001 e3530030
+ded4  3a000004 e3530037 9a0000f0 e3530064 1a000000 e38cc004 e0855002 eaffffae
+more>
+SP: 0xdad31db0:
+db0  60080093 c0951110 60080013 c0294b88 ffffffff bf11501c 40080013 ffffffff
+dd0  dad31e1c dca463c0 dad31e3c dad31de8 c000e318 c00083ac 00000000 00000000
+df0  de242b00 60080013 c09aaec0 bf115000 bf113068 00000001 dca463c0 dca463f0
+e10  00000000 dad31e3c dad319b8 dad31e30 c002de74 bf11501c 40080013 ffffffff
+e30  dad31e84 dad31e40 c0008728 bf11500c c0058d64 c0057604 dad30000 c0132c3c
+e50  dad31f48 bf113074 bf113068 dad31f48 bf113074 bf113068 00000001 dca463c0
+e70  dca463f0 00000001 dad31f44 dad31e88 c009c1d0 c0008620 bf113074 00007fff
+e90  c0097b90 dad31f4c dad31ec4 dad31ea8 c0053270 dad31edc 00000000 bf113074
+more>
+IP: 0xdad31938:
+  0000005a c0877214 c081f508 bf000000 dad319d4 dad31958 c000fc6c c000f440
+  00000000 c08023f4 c081f508 00000000 61542020 3a656c62 36633520 36303466
+  44202061 203a4341 30303030 35313030 c0db1300 0000049a de242b00 c0940540
+  dad319c4 dad319a8 c063e0f0 c00ae464 dad319fc dad31de8 c094c340 0000000f
+b8  c0db13f8 0000049a de242b00 c0940540 dad319ec dad319d8 c000fd0c c000f780
+d8  c0db11e8 c094c340 dad31a0c dad319f0 c00afdcc c000fcec 00000005 00000003
+f8  dad30000 c0db13f8 dad31a54 dad31a10 c00b33a0 c00afd98 dad31a2c 00000001
+a18  c001435c c02afc84 dad31a74 c00b60f8 00000060 c0db0ac4 dad31b14 00000000
+more>
+FP: 0xdad31dbc:
+dbc  c0294b88 ffffffff bf11501c 40080013 ffffffff dad31e1c dca463c0 dad31e3c
+ddc  dad31de8 c000e318 c00083ac 00000000 00000000 de242b00 60080013 c09aaec0
+dfc  bf115000 bf113068 00000001 dca463c0 dca463f0 00000000 dad31e3c dad319b8
+e1c  dad31e30 c002de74 bf11501c 40080013 ffffffff dad31e84 dad31e40 c0008728
+e3c  bf11500c c0058d64 c0057604 dad30000 c0132c3c dad31f48 bf113074 bf113068
+e5c  dad31f48 bf113074 bf113068 00000001 dca463c0 dca463f0 00000001 dad31f44
+e7c  dad31e88 c009c1d0 c0008620 bf113074 00007fff c0097b90 dad31f4c dad31ec4
+e9c  dad31ea8 c0053270 dad31edc 00000000 bf113074 bf1131b0 bf1131c4 dad30008
+more>
+R2: 0xde242a80:
+a80  00000000 dea21150 00000000 00000020 00000000 0000c350 0000c350 ffffffff
+aa0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
+ac0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
+ae0  00000000 00000001 ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
+b00  00000000 dad30000 00000002 00400100 00000000 00000000 00000001 00000001
+b20  00000078 00000078 00000078 00000000 c064f6f4 00000000 00000400 00400000
+b40  00000001 00000000 00000000 c1e936a0 c1e936a0 00000001 0856cb34 00000013
+b60  02c3c3fe 00000000 0bf3a24a 00000000 01714411 00000000 00000003 00000000
+more>
+R4: 0xc09aae40:
+ae40  c087fdb8 c088f1c0 c087fd6c c087fd4c c087fd6c c088f1c0 c087fd4c c087fd8c
+ae60  c087fd7c c087fd6c c087fda0 c087ffb8 c087ff94 c087ff78 c087ff54 c087ff30
+ae80  c087ff10 c087fe00 c087fe6c 00000000 00000000 00000000 00000000 00000000
+aea0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
+aec0  00000000 00000000 c07fe2c4 00000000 00000000 c1e69200 c1e692c0 00000000
+aee0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
+af00  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
+af20  00000001 00000000 00000000 00000000 00000000 00000000 003c7feb 00000000
+more>
+R8: 0xdca46340:
+  696e692e 65742e74 dc007478 00000000 00000000 00000000 00006e75 6f640112
+  6c65722d 65736165 6770752d 65646172 00000000 00000000 00000000 00000000
+more>
+Only 'q' or 'Q' are processed at more prompt, input ignored
+  00000000 bf113000 00002000 00000002 dca46940 00000001 00000000 c009a838
+a0  00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
+c0  dca46bc0 00000001 dca46980 00000124 c0dab510 00000000 00000000 00000024
+e0  bf113044 c0099200 00000000 00000000 00000000 00000000 00000000 00000000
+  00000100 dc5ef000 dca46da0 dca46d80 c0e4758c 00000000 00000000 c0886c5c
+  dca46420 dca46420 00000000 dac30400 0000002c 00000000 00000000 00000000
+R9: 0xdca46370:
+  00000000 00000000 00000000 00000000 00000000 bf113000 00002000 00000002
+  dca46940 00000001 00000000 c009a838 00000000 00000000 00000000 00000000
+b0  00000000 00000000 00000000 00000000 dca46bc0 00000001 dca46980 00000124
+d0  c0dab510 00000000 00000000 00000024 bf113044 c0099200 00000000 00000000
+f0  00000000 00000000 00000000 00000000 00000100 dc5ef000 dca46da0 dca46d80
+  c0e4758c 00000000 00000000 c0886c5c dca46420 dca46420 00000000 dac30400
+  0000002c 00000000 00000000 00000000 dca46580 b0d6a915 dca46889 00000000
+  00000000 00000000 0000171d 65640112 79732d62 6d657473 6e692d64 656b6f76
+dCPU: 4 PID: 1178 Comm: insmod Tainted: P           O 3.10.82-xu4hk #3
+Backtrace:
+[<c0012dec>] (dump_backtrace+0x0/0x114) from [<c001305c>] (show_stack+0x20/0x24)
+ r7:c0db13f8 r6:0000000f r5:c094c340 r4:dad31de8
+[<c001303c>] (show_stack+0x0/0x24) from [<c063ef68>] (dump_stack+0x24/0x28)
+[<c063ef44>] (dump_stack+0x0/0x28) from [<c000fd10>] (show_regs+0x30/0x34)
+[<c000fce0>] (show_regs+0x0/0x34) from [<c00afdcc>] (kdb_dumpregs+0x40/0x60)
+ r5:c094c340 r4:c0db11e8
+[<c00afd8c>] (kdb_dumpregs+0x0/0x60) from [<c00b33a0>] (kdb_main_loop+0x3bc/0x7b                                                                              4)
+ r7:c0db13f8 r6:dad30000 r5:00000003 r4:00000005
+[<c00b2fe4>] (kdb_main_loop+0x0/0x7b4) from [<c00b61d8>] (kdb_stub+0x26c/0x4d0)
+[<c00b5f6c>] (kdb_stub+0x0/0x4d0) from [<c00ac004>] (kgdb_cpu_enter+0x434/0x728)
+[<c00abbd0>] (kgdb_cpu_enter+0x0/0x728) from [<c00ac5fc>] (kgdb_handle_exception                                                                              +0x1ac/0x20c)
+[<c00ac450>] (kgdb_handle_exception+0x0/0x20c) from [<c0015590>] (kgdb_notify+0x                                                                              34/0x4c)
+ r9:00000000 r8:00000000 r7:fffffffd r6:00000001 r5:dad31bec
+r4:20080193
+[<c001555c>] (kgdb_notify+0x0/0x4c) from [<c0058908>] (notifier_call_chain+0x54/                                                                              0x94)
+ r5:dad31bec r4:c094e774
+[<c00588b4>] (notifier_call_chain+0x0/0x94) from [<c0058c18>] (__atomic_notifier                                                                              _call_chain+0x74/0xa0)
+ r9:00000000 r8:dad31bec r7:00000001 r6:c094e9cc r5:00000001
+r4:c094e774
+[<c0058ba4>] (__atomic_notifier_call_chain+0x0/0xa0) from [<c0059304>] (notify_d                                                                              ie+0x4c/0x54)
+more>
+Only 'q' or 'Q' are processed at more prompt, input ignored
+[<c00592b8>] (notify_die+0x0/0x54) from [<c001316c>] (die+0x10c/0x2ec)
+ r4:c09460b8
+[<c0013060>] (die+0x0/0x2ec) from [<c063dd1c>] (__do_kernel_fault.part.10+0x74/0                                                                              x84)
+[<c063dca8>] (__do_kernel_fault.part.10+0x0/0x84) from [<c001d040>] (do_page_fau                                                                              lt+0x3f4/0x400)
+ r7:de242b00 r4:dad30030
+[<c001cc4c>] (do_page_fault+0x0/0x400) from [<c001d1c4>] (do_translation_fault+0                                                                              xbc/0xc0)
+[<c001d108>] (do_translation_fault+0x0/0xc0) from [<c00083e8>] (do_DataAbort+0x4                                                                              8/0xac)
+ r7:00000000 r6:c001d108 r5:00000805 r4:c0946d3c
+[<c00083a0>] (do_DataAbort+0x0/0xac) from [<c000e318>] (__dabt_svc+0x38/0x60)
+Exception stack(0xdad31de8 to 0xdad31e30)
+de0:                   00000000 00000000 de242b00 60080013 c09aaec0 bf115000
+e00: bf113068 00000001 dca463c0 dca463f0 00000000 dad31e3c dad319b8 dad31e30
+e20: c002de74 bf11501c 40080013 ffffffff
+ r8:dca463c0 r7:dad31e1c r6:ffffffff r5:40080013 r4:bf11501c
+[<bf115000>] (my_oops_init+0x0/0x24 [code]) from [<c0008728>] (do_one_initcall+0                                                                              x114/0x174)
+[<c0008614>] (do_one_initcall+0x0/0x174) from [<c009c1d0>] (load_module+0x190c/0                                                                              x209c)
+more>
+Only 'q' or 'Q' are processed at more prompt, input ignored
+[<c009a8c4>] (load_module+0x0/0x209c) from [<c009cad4>] (SyS_finit_module+0x78/0                                                                              x88)
+[<c009ca5c>] (SyS_finit_module+0x0/0x88) from [<c000e7c0>] (ret_fast_syscall+0x0                                                                              /0x38)
+ r6:b6f76f10 r5:b6f8aad4 r4:4479ed00
+[4]kdb>
+[4]kdb>
+[4]kdb>
+[4]kdb>
+</code>
+**Host side :**
+Connection using gdb
+Initiate the ARM kgdb connection from the target machine to communicate with code.
+<code>
+# arm-linux-gnueabihf-gdb debug/code.ko
+GNU gdb (Linaro GDB 2014.11) 7.8-2014.09-1-git
+Copyright (C) 2014 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
+and "show warranty" for details.
+This GDB was configured as "--host=x86_64-unknown-linux-gnu --target=arm-linux-g                                                                              nueabihf".
+Type "show configuration" for configuration details.
+For bug reporting instructions, please see:
+<https://bugs.launchpad.net/gcc-linaro>.
+Find the GDB manual and other documentation resources online at:
+<http://www.gnu.org/software/gdb/documentation/>.
+For help, type "help".
+Type "apropos word" to search for commands related to "word"...
+Reading symbols from debug/code.ko...done.
+(gdb)
+(gdb) set debug remote 1
+(gdb) target remote /dev/ttyUSB1
+Remote debugging using /dev/ttyUSB1
+Sending packet: $qSupported:multiprocess+;qRelocInsn+#2a...Ack
+Saw new packet start in middle of old one
+Packet received:
+Packet qSupported (supported-packets) is NOT supported
+Sending packet: $Hg0#df...Ack
+Packet received: OK
+Sending packet: $qTStatus#49...Ack
+Packet received:
+Packet qTStatus (trace-status) is NOT supported
+Sending packet: $?#3f...Ack
+Packet received: S0b
+Sending packet: $qfThreadInfo#bb...Ack
+Packet received: mfffffffe,fffffffd,fffffffc,fffffffb,fffffffa,fffffff9,fffffff8                                                                              ,fffffff7,01,02,03,04,05,06,07,08,09,0a,0b,0c,0d,0e,0f,10,11
+Sending packet: $qAttached#8f...Ack
+Packet received:
+Packet qAttached (query-attached) is NOT supported
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m12,13,14,15,16,17,18,19,1a,1b,1c,1d,1e,1f,20,21,22
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m23,24,25,26,27,28,29,2a,2b,2c,2d,2e,2f,30,31,32,33
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m34,35,36,37,3a,3b,48,49,4a,4b,4c,4d,4e,4f,50,51,52
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m53,62,63,64,65,66,67,68,69,6a,6b,6c,6d,6e,75,76,77
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m78,79,7a,7b,7c,7d,a0,a1,a2,a3,a4,a5,a9,aa,c4,ec,f1
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m0125,013c,015a,0169,0172,0190,01a8,020f,0211,021e,0262,0212,02                                                                              5d,0264,0213,023b,0263
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m0214,0238,0239,023a,0215,0217,025c,0266,0267,021a,021c,0220,02                                                                              26,024a,0268,026c,0270
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m0274,028b,0289,028d,0295,029a,0291,0292,0298,0299,02a7,02b1,02                                                                              b2,02b5,02ba,02be,02bf
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m02c2,02c7,02c8,0304,031f,03a7,03f5,02c9,02cc,02d6,02ed,02f1,02                                                                              f0,02fe,0300,0306,0307
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m0326,032f,0335,0397,03a2,03a4,03a6,03e7,03a9,03d5,0403,0404,03                                                                              ab,03b1,03ac,03b3,03b8
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m03ad,03cc,03cd,03ce,03ae,03e0,03e1,03af,03c6,03c8,03f1,03b0,03                                                                              c2,03c3,03b6,03c5,03d2
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m03e2,0411,03d6,03dd,03de,03e5,03e9,03ea,03ed,03f3,0406,0408,04                                                                              09,046c,0472,0478,0485
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received: m048c,0497,0498,049a,049b
+Sending packet: $qsThreadInfo#c8...Ack
+Packet received:
+Sending packet: $Hc-1#09...Ack
+Packet received: OK
+Sending packet: $qC#b4...Ack
+Packet received: QC049a
+Sending packet: $qOffsets#4b...Ack
+Packet received:
+Sending packet: $g#67...Ack
+Packet received: 0000000000000000002b24de13000860c0ae9ac0005011bf683011bf0100000                                                                              0c063a4dcf063a4dc000000003c1ed3dab819d3da301ed3da74de02c01c5011bf000000000000000                                                                              00000000000000000000000000000000000000000000000000000000000000000
+Sending packet: $mbf11501c,4#f0...Ack
+Packet received: 000080e5
+Sending packet: $mbf115018,4#c5...Ack
+Packet received: 0000a0e3
+Sending packet: $mbf11501c,4#f0...Ack
+Packet received: 000080e5
+Sending packet: $mbf115018,4#c5...Ack
+Packet received: 0000a0e3
+Sending packet: $mbf11501c,2#ee...Ack
+Packet received: 0000
+Sending packet: $mbf11501a,2#ec...Ack
+Packet received: a0e3
+Sending packet: $mbf115018,2#c3...Ack
+Packet received: 0000
+Sending packet: $mbf11501c,2#ee...Ack
+Packet received: 0000
+Sending packet: $mbf11501a,2#ec...Ack
+Packet received: a0e3
+Sending packet: $mbf115018,2#c3...Ack
+Packet received: 0000
+Sending packet: $mbf11501c,4#f0...Ack
+Packet received: 000080e5
+Sending packet: $mbf115018,4#c5...Ack
+Packet received: 0000a0e3
+Sending packet: $mbf11501c,4#f0...Ack
+Packet received: 000080e5
+Sending packet: $mbf115018,4#c5...Ack
+Packet received: 0000a0e3
+Sending packet: $mbf11501c,4#f0...Ack
+Packet received: 000080e5
+Sending packet: $mbf11501c,4#f0...Ack
+Packet received: 000080e5
+Sending packet: $mbf11501c,4#f0...Ack
+Packet received: 000080e5
+Sending packet: $p19#da...Ack
+Packet received: 13000840
+Packet p (fetch-register) is supported
+Sending packet: $mdad31e3c,4#55...Ack
+Packet received: 0c5011bf
+xbf11501c in ?? ()
+Sending packet: $qSymbol::#5b...Ack
+Packet received:
+Packet qSymbol (symbol-lookup) is NOT supported
+(gdb)
+(gdb) disassemble my_oops_init
+(gdb) add-symbol-file debug/core.o 0x00000000
+add symbol table from file "debug/core.o" at
+        .text_addr = 0x0
+(y or n) y
+debug/core.o: No such file or directory.
+(gdb) add-symbol-file debug/core.ko 0x00000000
+add symbol table from file "debug/core.ko" at
+        .text_addr = 0x0
+(y or n) y
+debug/core.ko: No such file or directory.
+(gdb)
+(gdb) disassemble my_oops_init
+Dump of assembler code for function my_oops_init:
+x00000000 <+0>:     mov     r12, sp
+x00000004 <+4>:     push    {r11, r12, lr, pc}
+x00000008 <+8>:     sub     r11, r12, #4
+x0000000c <+12>:    movw    r0, #0
+x00000010 <+16>:    movt    r0, #0
+x00000014 <+20>:    bl      0x14 <my_oops_init+20>
+x00000018 <+24>:    mov     r0, #0
+x0000001c <+28>:    str     r0, [r0]
+x00000020 <+32>:    ldm     sp, {r11, sp, pc}
+End of assembler dump.
+(gdb)
+Dump of assembler code for function my_oops_init:
+x00000000 <+0>:     mov     r12, sp
+x00000004 <+4>:     push    {r11, r12, lr, pc}
+x00000008 <+8>:     sub     r11, r12, #4
+x0000000c <+12>:    movw    r0, #0
+x00000010 <+16>:    movt    r0, #0
+x00000014 <+20>:    bl      0x14 <my_oops_init+20>
+x00000018 <+24>:    mov     r0, #0
+x0000001c <+28>:    str     r0, [r0]
+x00000020 <+32>:    ldm     sp, {r11, sp, pc}
+End of assembler dump.
+</code>