0xAA - Random notes on security

Anti-debugging Techniques Cheat Sheet

It’s been quite a while I analyzed malware last time, so I decided to refresh my knowledge and write a short post on common x86 malware anti-debugging techniques. Techniques here do not include obfuscation like false branches, spaghetti code etc., and present an extract of popular ways to kick debugger’s ass. Please note: this is not a complete set of techniques and rather “shortcuts” than a guide. If you’d like to read more in details, I’ve provided links to some great antidbg materials in the end of the post. Feel free to contact me to complete the list with undescribed technique and/or correct already described ones!

Before we start, a little refreshment on breakpoints (OllyDbg has been taken as an example, although it’s true for most of debuggers):

  • software breakpoints - replacing original instruction with 0xCC and raising interrupt routine for debugger to handle it
  • hardware breakpoints - DR0…DR4 debug registers provided by the processor, as one of them is reached, INT 1 interrupt is raised by OS
  • memory breakpoints - guard pages, exception handler is called when accessing the specified page

Common anti-debugging techniques:

0) Straight checks for breakpoints

  • Detection of 0xCC bytes. Checks may include comparison to xor’ed value too, e.g. to 0x99 (0xCC ^ 0x55)
  • Detection of hardware breakpoints. Basically two methods are most common here. The first includes GetThreadContext/SetThreadContext API and direct check for DRs. Second method is used to set up own SEH, cause exception (i.e. xor eax,eax / div eax) and get direct access to debug registers in the handler as offset to context structure.
  • Detection of guard pages is somewhat rare and based on imitation of debugger behavior - i.e. creation of PAGE_GUARD memory page and accessing it, previously put return address onto the stack. If STATUS_GUARD_PAGE_VIOLATION occurs, it’s assumed no debugging is in place.

1) API calls

  • IsDebuggerPresent - probably the most well-known technique and one of the easiest to bypass. This API checks specific flag in PEB and returns TRUE/FALSE based on the result.
  • CheckRemoteDebuggerPresent - same functionality as previous - simple bool function, straight use
  • FindWindow - used to detect specific debuggers - for instance, OllyDbg window class is named “OLLYDBG” :) Other popular debuggers classes checks include “WinDbgFrameClass”, “ID”, “Zeta Debugger”, “Rock Debugger” and “ObsidianGUI”
  • NtQueryObject - detection is based on “debug objects”. API queries for the list of existing objects and checks the number of handles associated with any existing debug object
  • NtQuerySystemInformation (ZwQuerySystemInformation) - similar to previous point - checks if debug object handle exists and returns true if it’s the case
  • NtSetInformationThread (ZwSetInformationThread) - the first anti-debugging API implemented by Windows. Class HideThreadFromDebugger, when passed as an argument, can be used to prevent debuggers from receiving events (include breakpoints and exiting the program) from any thread that has this API called on it.
  • NtContinue and similar functions are used modify current context or load a new one in the current thread, which can confuse debugger.
  • CloseHandle and NtClose - a very cool technique based on the fact that call of ZwClose with invalid handle generates STATUS_INVALID_HANDLE exception when the process is debugged.
  • GenerateConsoleCtrlEvent - event-based detection. One vector is to invoke Ctrl-C signal and check for EXCEPTION_CTL_C exception (which is true if the process is debugged)
  • OutputDebugString with a valid ASCII strings - causes error when no debugger is present, otherwise passes normally. Can also be used to exploit known weaknesses - for example, OllyDbg had known bug of not correct handling of format strings and crashed with multiple “%s” input.
  • seems this list can be extended ad infinitum…

2) Flags

  • Trap flag - controls tracing of a program. If it’s set, executing an instruction will raise SINGLE_STEP exception. Example of usage: pushf / mov dword [esp], 0x100 / popf. Another possible scenario might be tracing over SS (stack segment register) - debugger will not break on those (e.g. push ss / pop ss) effectively stopping on the following instruction. In other words, unset of trapflag won’t be possible after that, and if check is done here, debugger will be detected.
  • IsDebugged - second byte of PEB - this is what checked by IsDebuggerPresent(), however, can also be checked directly.
  • NtGlobalFlag - another field in PEB with offset 0x68/0xBC (x86/x64). A process that is created by debugger will have 0x70 value (FLG_HEAP_ENABLE_TAIL_CHECK | FLG_HEAP_ENABLE_FREE_CHECK | FLG_HEAP_VALIDATE_PARAMETERS) by default
  • Heap flags - check of two flags located in heap: “Flags” and “ForceFlags”. Normally heap location can be retrieved by GetProcessHeap() and/or from PEB structure. Exact combination of flags depend on the OS (see more in details following links at the bottom)

3) Timing

  • GetTickCount, GetLocalTime, GetSystemTime, timeGetTime, NtQueryPerformanceCounter - typical timing functions which are used to measure time needed to execute some function / instruction set. If difference is more than fixed threshold, the process exits.
  • rdtsc - “Read Time Stamp Counter” asm instruction,technique is the same as described above

4) Checksums

This method is based on calculation of CRC32 for certain blocks or whole binary and comparing to hardcoded value. If values differ, it indicates dynamic code changes were made (breakpoints/patches), and the process usually exits.

5) Self-debug

There are different approaches for this, probably the most recongnized one is to create a new process and call DebugActiveProcess(pid) on the parent process. If the process is already being debugged, associated syscall ZwDebugActiveProcess() will fail, making it clear something is wrong :)

6) Rogue instructions

  • INT3 - classic example (0xCC, 0xCD+0x03). Checks may include comparison to xor’ed value, e.g. to 0x99 (0xCC ^ 0x55)
  • Single-step - old trick to insert 0xF1 opcode to exploit SoftICE debugging process by generating SINGLE_STEP exception.
  • INT 2Dh - powerful interrupt technique which results in raising breakpoint exception if the process is not debugged and in normal execution if debugger is present.
  • Stack Segment register - already described in “Trap flag” section - due to incorrect execution of SS registers, it is possible to trick the debugger setting the flag and check its value immediately.

7) Bonus:

The best protection against debugging so far seems to be own virtual machine. Effectively, part of object code is converted to self bytecode format, which is run on a self-written VM. The only way to properly debug such code will be emulator/disassembler for custom VM instruction format.


More to read:
http://pferrie.host22.com/papers/antidebug.pdf
http://thelegendofrandom.com/blog/archives/2100
http://www.symantec.com/connect/articles/windows-anti-debug-reference
http://spareclockcycles.org/2012/02/14/stack-necromancy-defeating-debuggers-by-raising-the-dead/