This book is dedicated to Robert Jourdain, John Socha, Ralf Brown and Peter Abel 52 65 76 65 72 73 65 45 6e 67 69 6e 65 65 72 69 6e 67 66 6f 72 20 42 65 67 69 6e 6e 65 72 73 44 65 6e 6e 69 73 20 59 75 72 69 63 68 65 76 Reverse Engineering for Beginners Dennis Yurichev c ba ©2013-2016, Dennis Yurichev This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license To view a copy of this license, visit https://creativecommons.org/licenses/by-sa/4.0/ Text version (April 7, 2018) The latest version (and Russian edition) of this text is accessible at beginners.re The cover was made by Andy Nechaevsky: facebook i Call for translators! You may want to help me with translating this work into languages other than English and Russian Just send me any piece of translated text (no matter how short) and I’ll put it into my LaTeX source code Read here We already have something in German, French, a bit in Italian, Portuguese and Polish Speed isn’t important, because this is an open-source project, after all Your name will be mentioned as a project contributor Korean, Chinese, and Persian languages are reserved by publishers English and Russian versions I by myself, but my English is still that horrible, so I’m very grateful for any notes about grammar, etc Even my Russian is flawed, so I’m grateful for notes about Russian text as well! So not hesitate to contact me: dennis@yurichev.com ii Abridged contents Code Patterns Important fundamentals 490 Slightly more advanced examples 527 Java 761 Finding important/interesting stuff in the code 801 OS-specific 855 Tools 918 Case studies 928 Examples of reversing proprietary file formats 1083 10 Other things 1158 11 Books/blogs worth reading 1184 12 Communities 1188 Afterword 1190 Appendix 1193 Acronyms used 1231 Glossary 1236 Index 1238 iii CONTENTS Contents Code Patterns 1.1 The method 1.2 Some basics 1.2.1 A short introduction to the CPU 1.2.2 Numeral Systems 1.2.3 Converting From One Radix To Another 1.3 An Empty Function 1.3.1 x86 1.3.2 ARM 1.3.3 MIPS 1.3.4 Empty Functions in Practice 1.4 Returning Values 1.4.1 x86 1.4.2 ARM 1.4.3 MIPS 1.5 Hello, world! 1.5.1 x86 1.5.2 x86-64 1.5.3 GCC—one more thing 1.5.4 ARM 1.5.5 MIPS 1.5.6 Conclusion 1.5.7 Exercises 1.6 Function prologue and epilogue 1.6.1 Recursion 1.7 Stack 1.7.1 Why does the stack grow backwards? 1.7.2 What is the stack used for? 1.7.3 A typical stack layout 1.7.4 Noise in stack 1.7.5 Exercises 1.8 printf() with several arguments 1.8.1 x86 1.8.2 ARM 1.8.3 MIPS 1.8.4 Conclusion 1.8.5 By the way 1.9 scanf() 1.9.1 Simple example 1.9.2 Popular mistake 1.9.3 Global variables 1.9.4 scanf() 1.9.5 Exercise 1.10 Accessing passed arguments 1.10.1 x86 1.10.2 x64 1.10.3 ARM 1.10.4 MIPS 1.11 More about results returning 1.11.1 Attempt to use the result of a function returning void 1.11.2 What if we not use the function result? 1.11.3 Returning a structure 1.12 Pointers iv 5 6 10 10 10 10 11 13 13 13 13 16 16 21 25 26 32 36 36 38 38 40 40 41 47 47 51 53 53 64 69 75 76 79 79 88 89 98 109 111 111 113 116 119 122 122 123 123 126 CONTENTS 1.12.1 Swap input values 1.12.2 Returning values 1.13 GOTO operator 1.13.1 Dead code 1.13.2 Exercise 1.14 Conditional jumps 1.14.1 Simple example 1.14.2 Calculating absolute value 1.14.3 Ternary conditional operator 1.14.4 Getting minimal and maximal values 1.14.5 Conclusion 1.14.6 Exercise 1.15 switch()/case/default 1.15.1 Small number of cases 1.15.2 A lot of cases 1.15.3 When there are several case statements in one block 1.15.4 Fall-through 1.15.5 Exercises 1.16 Loops 1.16.1 Simple example 1.16.2 Memory blocks copying routine 1.16.3 Condition check 1.16.4 Conclusion 1.16.5 Exercises 1.17 More about strings 1.17.1 strlen() 1.17.2 Boundaries of strings 1.18 Replacing arithmetic instructions to other ones 1.18.1 Multiplication 1.18.2 Division 1.18.3 Exercise 1.19 Floating-point unit 1.19.1 IEEE 754 1.19.2 x86 1.19.3 ARM, MIPS, x86/x64 SIMD 1.19.4 C/C++ 1.19.5 Simple example 1.19.6 Passing floating point numbers via arguments 1.19.7 Comparison example 1.19.8 Some constants 1.19.9 Copying 1.19.10 Stack, calculators and reverse Polish notation 1.19.11 80 bits? 1.19.12 x64 1.19.13 Exercises 1.20 Arrays 1.20.1 Simple example 1.20.2 Buffer overflow 1.20.3 Buffer overflow protection methods 1.20.4 One more word about arrays 1.20.5 Array of pointers to strings 1.20.6 Multidimensional arrays 1.20.7 Pack of strings as a two-dimensional array 1.20.8 Conclusion 1.21 By the way 1.21.1 Exercises 1.22 Manipulating specific bit(s) 1.22.1 Specific bit checking 1.22.2 Setting and clearing specific bits 1.22.3 Shifts 1.22.4 Setting and clearing specific bits: FPU1 example 1.22.5 Counting bits set to 1.22.6 Conclusion Floating-Point Unit v 126 126 138 140 141 143 143 160 162 165 170 171 173 173 186 198 202 203 205 205 216 219 220 221 223 223 234 236 236 241 241 243 243 243 243 243 244 255 258 292 292 292 292 292 292 294 294 301 309 312 313 320 326 330 332 332 334 334 337 346 346 350 365 CONTENTS 1.22.7 Exercises 1.23 [ 1.23.1 x86 1.23.2 x64 1.23.3 32-bit ARM 1.23.4 MIPS 1.23.5 Thread-safe version of the example 1.24 Structures 1.24.1 MSVC: SYSTEMTIME example 1.24.2 Let’s allocate space for a structure using malloc() 1.24.3 UNIX: struct tm 1.24.4 Fields packing in structure 1.24.5 Nested structures 1.24.6 Bit fields in a structure 1.24.7 Exercises 1.25 Unions 1.25.1 Pseudo-random number generator example 1.25.2 Calculating machine epsilon 1.26 FSCALE replacement 1.26.1 Fast square root calculation 1.27 Pointers to functions 1.27.1 MSVC 1.27.2 GCC 1.27.3 Danger of pointers to functions 1.28 64-bit values in 32-bit environment 1.28.1 Returning of 64-bit value 1.28.2 Arguments passing, addition, subtraction 1.28.3 Multiplication, division 1.28.4 Shifting right 1.28.5 Converting 32-bit value into 64-bit one 1.29 SIMD 1.29.1 Vectorization 1.29.2 SIMD strlen() implementation 1.30 64 bits 1.30.1 x86-64 1.30.2 ARM 1.30.3 Float point numbers 1.30.4 64-bit architecture criticism 1.31 Working with floating point numbers using SIMD 1.31.1 Simple example 1.31.2 Passing floating point number via arguments 1.31.3 Comparison example 1.31.4 Calculating machine epsilon: x64 and SIMD 1.31.5 Pseudo-random number generator example revisited 1.31.6 Summary 1.32 ARM-specific details 1.32.1 Number sign (#) before number 1.32.2 Addressing modes 1.32.3 Loading a constant into a register 1.32.4 Relocs in ARM64 1.33 MIPS-specific details 1.33.1 Loading a 32-bit constant into register 1.33.2 Further reading about MIPS 367 369 369 370 371 371 374 376 376 380 381 391 398 401 408 410 410 414 418 419 421 422 428 432 434 434 435 438 441 443 445 445 455 460 460 466 466 467 469 469 476 477 479 480 480 483 483 483 484 485 488 488 489 Important fundamentals 2.1 Integral datatypes 2.1.1 Bit 2.1.2 Nibble AKA nybble 2.1.3 Byte 2.1.4 Wide char 2.1.5 Signed integer vs unsigned 2.1.6 Word 2.1.7 Address register 2.1.8 Numbers 490 492 492 492 493 494 494 494 495 496 vi CONTENTS 2.2 Signed number representations 2.2.1 Using IMUL over MUL 2.2.2 Couple of additions about two’s complement form 2.3 Integer overflow 2.4 AND 2.4.1 Checking if a value is on 2n boundary 2.4.2 KOI-8R Cyrillic encoding 2.5 AND and OR as subtraction and addition 2.5.1 ZX Spectrum ROM text strings 2.6 XOR (exclusive OR) 2.6.1 Everyday speech 2.6.2 Encryption 2.6.3 RAID2 2.6.4 XOR swap algorithm 2.6.5 XOR linked list 2.6.6 Zobrist hashing / tabulation hashing 2.6.7 By the way 2.6.8 AND/OR/XOR as MOV 2.7 Population count 2.8 Endianness 2.8.1 Big-endian 2.8.2 Little-endian 2.8.3 Example 2.8.4 Bi-endian 2.8.5 Converting data 2.9 Memory 2.10 CPU 2.10.1 Branch predictors 2.10.2 Data dependencies 2.11 Hash functions 2.11.1 How one-way functions work? 499 500 501 503 506 506 506 509 509 513 513 513 513 513 514 514 515 515 517 519 519 519 519 520 520 522 524 524 524 526 526 Slightly more advanced examples 3.1 Double negation 3.2 strstr() example 3.3 Temperature converting 3.3.1 Integer values 3.3.2 Floating-point values 3.4 Fibonacci numbers 3.4.1 Example #1 3.4.2 Example #2 3.4.3 Summary 3.5 CRC32 calculation example 3.6 Network address calculation example 3.6.1 calc_network_address() 3.6.2 form_IP() 3.6.3 print_as_IP() 3.6.4 form_netmask() and set_bit() 3.6.5 Summary 3.7 Loops: several iterators 3.7.1 Three iterators 3.7.2 Two iterators 3.7.3 Intel C++ 2011 case 3.8 Duff’s device 3.8.1 Should one use unrolled loops? 3.9 Division using multiplication 3.9.1 x86 3.9.2 How it works 3.9.3 ARM 3.9.4 MIPS 3.9.5 Exercise 3.10 String to number conversion (atoi()) 3.10.1 Simple example 527 529 531 533 533 535 539 539 542 546 549 553 554 554 556 557 558 560 560 561 562 565 567 569 569 570 570 572 572 574 574 Redundant Array of Independent Disks vii CONTENTS 3.10.2 A slightly advanced example 3.10.3 Exercise 3.11 Inline functions 3.11.1 Strings and memory functions 3.12 C99 restrict 3.13 Branchless abs() function 3.13.1 Optimizing GCC 4.9.1 x64 3.13.2 Optimizing GCC 4.9 ARM64 3.14 Variadic functions 3.14.1 Computing arithmetic mean 3.14.2 vprintf() function case 3.14.3 Pin case 3.14.4 Format string exploit 3.15 Strings trimming 3.15.1 x64: Optimizing MSVC 2013 3.15.2 x64: Non-optimizing GCC 4.9.1 3.15.3 x64: Optimizing GCC 4.9.1 3.15.4 ARM64: Non-optimizing GCC (Linaro) 4.9 3.15.5 ARM64: Optimizing GCC (Linaro) 4.9 3.15.6 ARM: Optimizing Keil 6/2013 (ARM mode) 3.15.7 ARM: Optimizing Keil 6/2013 (Thumb mode) 3.15.8 MIPS 3.16 toupper() function 3.16.1 x64 3.16.2 ARM 3.16.3 Using bit operations 3.16.4 Summary 3.17 Obfuscation 3.17.1 Text strings 3.17.2 Executable code 3.17.3 Virtual machine / pseudo-code 3.17.4 Other things to mention 3.17.5 Exercise 3.18 C++ 3.18.1 Classes 3.18.2 ostream 3.18.3 References 3.18.4 STL 3.18.5 Memory 3.19 Negative array indices 3.19.1 Addressing string from the end 3.19.2 Addressing some kind of block from the end 3.19.3 Arrays started at 3.20 [ 3.20.1 Introduction 3.20.2 Data structure 3.20.3 The algorithm 3.20.4 The C/C++ code 3.20.5 How it works 3.20.6 Optimizing GCC 4.8.2 for x86-64 3.20.7 Optimizing Keil 5.05 (Thumb mode) 3.20.8 Optimizing Keil 5.05 (ARM mode) 3.20.9 (32-bit ARM) Comparison of code density in Thumb and ARM modes 3.20.10 Optimizing GCC 4.9.3 for ARM64 3.20.11 Optimizing GCC 4.4.5 for MIPS 3.20.12 Difference from the real FAT12 3.20.13 Exercise 3.20.14 Summary 3.20.15 Conclusion 3.21 More about pointers 3.21.1 Working with addresses instead of pointers 3.21.2 Passing values as pointers; tagged unions 3.21.3 Pointers abuse in Windows kernel 3.21.4 Null pointers viii 577 580 582 582 592 596 596 596 599 599 603 604 604 607 607 609 610 611 612 613 613 614 617 617 619 620 621 623 623 623 625 625 625 627 627 643 644 645 678 680 680 680 680 684 684 684 684 685 687 688 690 692 694 694 696 698 699 699 699 701 701 703 704 709 Acronyms used 1230 .6 CHEATSHEETS OS Operating System xvi OOP Object-Oriented Programming .628 PL Programming Language xiv PRNG Pseudorandom Number Generator ix ROM Read-Only Memory 94 ALU Arithmetic Logic Unit 33 PID Program/process ID 946 LF Line Feed (10 or ’\n’ in C/C++) 607 CR Carriage Return (13 or ’\r’ in C/C++) 607 LIFO Last In First Out 40 MSB Most Significant Bit 346 LSB Least Significant Bit NSA National Security Agency 517 CFB Cipher Feedback 1014 CSPRNG Cryptographically Secure Pseudorandom Number Generator 1015 SICP Structure and Interpretation of Computer Programs .xvii ABI Application Binary Interface 22 RA Return Address 29 PE Portable Executable SP stack pointer SP/ESP/RSP in x86/x64 SP in ARM 26 DLL Dynamic-Link Library 885 PC Program Counter IP/EIP/RIP in x86/64 PC in ARM 27 LR Link Register 10 IDA Interactive Disassembler and Debugger developed by Hex-Rays 10 IAT Import Address Table .885 INT Import Name Table .886 1231 .6 CHEATSHEETS RVA Relative Virtual Address 885 VA Virtual Address 885 OEP Original Entry Point 871 MSVC Microsoft Visual C++ MSVS Microsoft Visual Studio 1209 ASLR Address Space Layout Randomization 711 MFC Microsoft Foundation Classes 889 TLS Thread Local Storage .310 AKA Also Known As 40 CRT C Runtime library 17 CPU Central Processing Unit xvi GPU Graphics Processing Unit 1026 FPU Floating-Point Unit v CISC Complex Instruction Set Computing .27 RISC Reduced Instruction Set Computing GUI Graphical User Interface 882 RTTI Run-Time Type Information 643 BSS Block Started by Symbol 32 SIMD Single Instruction, Multiple Data 216 BSOD Blue Screen of Death 873 DBMS Database Management Systems xiv ISA Instruction Set Architecture x HPC High-Performance Computing .594 SEH Structured Exception Handling 46 ELF Executable File format widely used in *NIX systems including Linux 92 TIB Thread Information Block .310 1232 .6 CHEATSHEETS PIC Position Independent Code 623 NAN Not a Number 1200 NOP No Operation 10 BEQ (PowerPC, ARM) Branch if Equal 108 BNE (PowerPC, ARM) Branch if Not Equal 231 BLR (PowerPC) Branch to Link Register .958 XOR eXclusive OR .1206 MCU Microcontroller Unit 566 RAM Random-Access Memory .6 GCC GNU Compiler Collection EGA Enhanced Graphics Adapter 1173 VGA Video Graphics Array .1173 API Application Programming Interface 723 ASCII American Standard Code for Information Interchange 320 ASCIIZ ASCII Zero (null-terminated ASCII string ) 105 IA64 Intel Architecture 64 (Itanium) 520 EPIC Explicitly Parallel Instruction Computing 1169 OOE Out-of-Order Execution 524 MSDN Microsoft Developer Network 714 STL (C++) Standard Template Library 649 PODT (C++) Plain Old Data Type 661 HDD Hard Disk Drive 672 VM Virtual Memory WRK Windows Research Kernel 830 GPR General Purpose Registers SSDT System Service Dispatch Table 873 1233 .6 CHEATSHEETS RE Reverse Engineering 1188 RAID Redundant Array of Independent Disks vii BCD Binary-Coded Decimal 492 BOM Byte Order Mark 813 GDB GNU Debugger 59 FP Frame Pointer 31 MBR Master Boot Record 822 JPE Jump Parity Even (x86 instruction) 264 CIDR Classless Inter-Domain Routing 553 STMFD Store Multiple Full Descending (ARM instruction) LDMFD Load Multiple Full Descending (ARM instruction) STMED Store Multiple Empty Descending (ARM instruction) .40 LDMED Load Multiple Empty Descending (ARM instruction) 40 STMFA Store Multiple Full Ascending (ARM instruction) 40 LDMFA Load Multiple Full Ascending (ARM instruction) 40 STMEA Store Multiple Empty Ascending (ARM instruction) 40 LDMEA Load Multiple Empty Ascending (ARM instruction) 40 APSR (ARM) Application Program Status Register 287 FPSCR (ARM) Floating-Point Status and Control Register 287 RFC Request for Comments 818 TOS Top of Stack .764 LVA (Java) Local Variable Array 770 JVM Java Virtual Machine ix JIT Just-In-Time compilation 763 CDFS Compact Disc File System 839 CD Compact Disc 1234 .6 CHEATSHEETS ADC Analog-to-Digital Converter 835 EOF End of File 98 DIY Do It Yourself 717 MMU Memory Management Unit 710 DES Data Encryption Standard 493 MIME Multipurpose Internet Mail Extensions .493 DBI Dynamic Binary Instrumentation 604 XML Extensible Markup Language 728 JSON JavaScript Object Notation 728 URL Uniform Resource Locator IV Initialization Vector xi RSA Rivest Shamir Adleman 1116 CPRNG Cryptographically secure PseudoRandom Number Generator 1117 GiB Gibibyte 1134 1235 Glossary heap usually, a big chunk of memory provided by the OS so that applications can divide it by themselves as they wish malloc()/free() work with the heap 40, 380, 646, 648, 661, 662, 677, 678, 728, 884, 885 real number numbers which may contain a dot this is float and double in C/C++ 241 decrement Decrease by 26, 205, 225, 483, 842, 1001, 1202, 1205, 1209 increment Increase by 23, 27, 205, 209, 225, 231, 355, 358, 483, 998, 1202 integral data type usual numbers, but not a real ones may be used for passing variables of boolean data type and enumerations 257 product Multiplication result 111, 249, 252, 446, 473, 500 arithmetic mean a sum of all values divided by their count 599 stack pointer A register pointing to a place in the stack 17, 18, 27, 40, 44, 53, 65, 66, 86, 113, 630, 751, 857–859, 1197, 1203, 1216, 1231 tail call It is when the compiler (or interpreter) transforms the recursion (with which it is possible: tail recursion) into an iteration for efficiency : wikipedia 546 quotient Division result 241, 245, 247, 248, 252, 472, 569, 600 anti-pattern Generally considered as bad practice 42, 89, 522 atomic operation “ατ oµoς” stands for “indivisible” in Greek, so an atomic operation is guaranteed not to be interrupted by other threads 744, 917 basic block a group of instructions that not have jump/branch instructions, and also don’t have jumps inside the block from the outside In IDA it looks just like as a list of instructions without empty lines 793, 1175, 1176 callee A function being called by another 42, 57, 79, 99, 111, 113, 115, 460, 522, 630, 751, 857–859, 862, 863, 1220 caller A function calling another 10, 13, 17, 38, 57, 99, 111, 112, 114, 123, 175, 460, 533, 630, 857, 859, 863 compiler intrinsic A function specific to a compiler which is not an usual library function The compiler generates a specific machine code instead of a call to it Often, it’s a pseudofunction for a specific CPU instruction Read more: ( 10.3 on page 1164) 1209 CP/M Control Program for Microcomputers: a very basic disk OS used before MS-DOS 1064 dongle Dongle is a small piece of hardware connected to LPT printer port (in past) or to USB Its function was similar to a security token, it has some memory and, sometimes, a secret (crypto-)hashing algorithmi 957 endianness Byte order: 2.8 on page 519 29, 91, 378, 1206 GiB Gibibyte: 230 or 1024 mebibytes or 1073741824 bytes 22 jump offset a part of the JMP or Jcc instruction’s opcode, to be added to the address of the next instruction, and this is how the new PC is calculated May be negative as well 106, 152, 1202 1236 Glossary kernel mode A restrictions-free CPU mode in which the OS kernel and drivers execute cf 1237 user mode leaf function A function which does not call any other function 35, 42 link register (RISC) A register where the return address is usually stored This makes it possible to call leaf functions without using the stack, i.e., faster 41, 958, 1216, 1217 loop unwinding It is when a compiler, instead of generating loop code for n iterations, generates just n copies of the loop body, in order to get rid of the instructions for loop maintenance 207 name mangling used at least in C++, where the compiler needs to encode the name of class, method and argument types in one string, which will become the internal name of the function You can read more about it here : 3.18.1 on page 628 628, 803, 804 NaN not a number: a special cases for floating point numbers, usually signaling about errors 260, 282, 1171 NEON AKA “Advanced SIMD”—SIMD from ARM 1217 NOP “no operation”, idle instruction 842 NTAPI API available only in the Windows NT line Largely not documented by Microsoft 931 padding Padding in English language means to stuff a pillow with something to give it a desired (bigger) form In computer science, padding means to add more bytes to a block so it will have desired size, like 2n bytes 815, 816 PDB (Win32) Debugging information file, usually just function names, but sometimes also function arguments and local variables names 801, 887, 931, 932, 941, 1045 POKE BASIC language instruction for writing a byte at a specific address 842 register allocator The part of the compiler that assigns CPU registers to local variables 224, 336, 460 reverse engineering act of understanding how the thing works, sometimes in order to clone it 1209 iv, security cookie A random value, different at each execution You can read more about it here : 1.20.3 on page 309 907 stack frame A part of the stack that contains information specific to the current function: local variables, function arguments, RA, etc 80, 111, 112, 542, 907 stdout standard output 29, 45, 175 thunk function Tiny function with a single role: call another function 30, 429, 958, 966 tracer My own simple debugging tool You can read more about it here : 7.2.1 on page 922 210–212, 810, 826, 832, 834, 903, 912, 1047, 1054, 1058, 1059, 1061, 1160 user mode A restricted CPU mode in which it all application software code is executed cf 974, 1237 kernel mode Windows NT Windows NT, 2000, XP, Vista, 7, 8, 10 319, 456, 750, 814, 873, 886, 916, 1069, 1208 word data type fitting in GPR In the computers older than PCs, the memory size was often measured in words rather than bytes 492–495, 501, 651, 730 xoring often used in the English language, which implying applying the XOR operation 907, 969, 972 1237 Index NET, 892 0x0BADF00D, 89 0xCCCCCCCC, 89 Ada, 119 AES, 1012 Alpha AXP, AMD, 862 Angry Birds, 288, 289 Apollo Guidance Computer, 233 Apophenia, 851 ARM, 231, 847, 957, 1216 Addressing modes, 483 ARM mode, ARM1, 495 armel, 253 armhf, 253 Condition codes, 155 D-registers, 252, 1217 Data processing instructions, 571 DCB, 27 hard float, 253 if-then block, 288 Instructions ADC, 437 ADD, 28, 119, 155, 213, 350, 362, 571, 1218 ADDAL, 155 ADDCC, 194 ADDS, 117, 437, 1218 ADR, 26, 155 ADRcc, 155, 183, 524 ADRP/ADD pair, 31, 66, 95, 316, 329, 485 ANDcc, 619 ASR, 365 ASRS, 344, 571 B, 65, 155, 156 Bcc, 108, 109, 167 BCS, 156, 290 BEQ, 107, 183 BGE, 156 BIC, 344, 349, 367 BL, 27–31, 155, 486 BLcc, 155 BLE, 156 BLS, 156 BLT, 213 BLX, 29 BNE, 156 BX, 117, 196 CMP, 107, 108, 155, 183, 194, 213, 362, 1218 CSEL, 164, 169, 171, 363 EOR, 349 FCMPE, 290 FCSEL, 290 1238 FMOV, 485 FMRS, 350 IT, 171, 288, 312 LDMccFD, 155 LDMEA, 40 LDMED, 40 LDMFA, 40 LDMFD, 27, 40, 155 LDP, 32 LDR, 67, 86, 94, 298, 315, 483 LDRB, 396 LDRB.W, 231 LDRSB, 231 LEA, 524 LSL, 362, 365 LSL.W, 362 LSLR, 619 LSLS, 299, 349, 619 LSR, 365 LSRS, 349 MADD, 117 MLA, 116, 117 MOV, 13, 27, 28, 362, 571 MOVcc, 167, 171 MOVK, 484 MOVT, 28, 571 MOVT.W, 29 MOVW, 29 MUL, 119 MULS, 117 MVNS, 231 NEG, 580 ORR, 344 POP, 26–28, 40, 42 PUSH, 28, 40, 42 RET, 32 RSB, 161, 325, 362, 580 SBC, 437 SMMUL, 571 STMEA, 40 STMED, 40 STMFA, 40, 68 STMFD, 26, 40 STMIA, 67 STMIB, 68 STP, 31, 66 STR, 66, 298 SUB, 66, 325, 362 SUBcc, 619 SUBEQ, 232 SUBS, 437 SXTB, 397 SXTW, 329 INDEX TEST, 224 TST, 337, 362 VADD, 252 VDIV, 252 VLDR, 252 VMOV, 252, 287 VMOVGT, 287 VMRS, 287 VMUL, 252 XOR, 161, 350 Leaf function, 42 Mode switching, 117, 196 mode switching, 29 Optional operators ASR, 362, 571 LSL, 298, 325, 362, 484 LSR, 362, 571 ROR, 362 RRX, 362 Pipeline, 194 Registers APSR, 287 FPSCR, 287 Link Register, 27, 41, 65, 196, 1216 R0, 122, 1216 scratch registers, 231, 1216 X0, 1217 Z, 108, 1216 S-registers, 252, 1217 soft float, 253 Thumb mode, 5, 156, 195 Thumb-2 mode, 5, 195, 287, 289 ARM64 lo12, 66 ASLR, 886 AT&T syntax, 19, 46 AWK, 829 Base address, 885 base32, 816 Base64, 815 base64, 818, 1009, 1118 base64scanner, 517, 816 bash, 122 BASIC POKE, 842 BeagleBone, 1022 binary grep, 824, 920 Binary Ninja, 920 Binary tree, 668 BIND.EXE, 891 BinNavi, 920 binutils, 414 Binwalk, 1110 Bitcoin, 741, 1022 Booth’s multiplication algorithm, 240 Borland C++, 709 Borland C++Builder, 804 Borland Delphi, 804, 812, 1160, 1210 BSoD, 873 BSS, 887 Buffer Overflow, 301, 308, 907 C language elements C99, 124 bool, 334 restrict, 592 variable length arrays, 312 Comma, 1182 const, 16, 94 for, 205, 550 if, 143, 174 Pointers, 79, 86, 126, 421, 460, 701 Post-decrement, 483 Post-increment, 483 Pre-decrement, 483 Pre-increment, 483 ptrdiff_t, 716 return, 17, 99, 123 Short-circuit, 607, 610, 1182 switch, 173, 174, 183 while, 223 C standard library alloca(), 44, 312, 522, 898 assert(), 318, 820 atexit(), 650 atoi(), 574, 1035 calloc(), 991 close(), 879 exit(), 533 fread(), 725 free(), 522, 678 fseek(), 991 ftell(), 991 fwrite(), 725 getenv(), 1036 localtime(), 759 localtime_r(), 387 longjmp, 732 longjmp(), 175 malloc(), 380, 522, 678 memchr(), 1205 memcmp(), 497, 589, 822, 1206 memcpy(), 19, 79, 586, 730, 1204 memmove(), 730 memset(), 292, 585, 1058, 1205, 1206 open(), 879 pow(), 255 puts(), 28 qsort(), 421 rand(), 369, 809, 937, 941, 978 read(), 725, 879 realloc(), 522 scanf(), 79 setjmp, 732 strcat(), 590 strcmp(), 497, 583, 879 strcpy(), 19, 585, 978 strlen(), 223, 455, 585, 608, 1205 strstr(), 531 strtok, 234 time(), 759 tolower(), 996 toupper(), 617 va_arg, 599 va_list, 603 vprintf, 603 write(), 725 C++, 1048 1239 INDEX C++11, 661, 867 exceptions, 898 ostream, 643 References, 644 RTTI, 643 STL, 801 std::forward_list, 660 std::list, 651 std::map, 668 std::set, 668 std::string, 645 std::vector, 661 C11, 867 Callbacks, 421 Canary, 309 cdecl, 53, 857 Chess, 514 Cipher Feedback mode, 1014 clusterization, 1115 COFF, 964 column-major order, 320 Compiler intrinsic, 45, 500, 1164 Compiler’s anomalies, 166, 255, 328, 344, 361, 562, 613, 1166 Cray-1, 495 CRC32, 526, 549 CRT, 882, 904 CryptoMiniSat, 467 CryptoPP, 1012 Cygwin, 803, 810, 892, 922 Data general Nova, 240 DES, 445, 460 dlopen(), 879 dlsym(), 879 Donald E Knuth, 495 DOSBox, 1069 DosBox, 832 double, 243, 862 Doubly linked list, 514, 651 dtruss, 922 Duff’s device, 565 Dynamically loaded libraries, 29 Edsger W Dijkstra, 680 EICAR, 1064 ELF, 92 Entropy, 1087, 1107 Error messages, 817 fastcall, 21, 43, 76, 335, 858 fetchmail, 493 FidoNet, 816 float, 243, 862 Forth, 785 FORTRAN, 30 Fortran, 320, 592, 680, 803 FreeBSD, 823 Function epilogue, 38, 65, 67, 155, 396, 829 Function prologue, 18, 38, 42, 66, 309, 829 Fused multiply–add, 116, 117 Fuzzing, 580 Garbage collector, 786 GCC, 803, 1223, 1228 GDB, 36, 59, 62, 308, 429, 430, 922, 1228 GeoIP, 1108 GHex, 920 Glibc, 429, 730, 873 Global variables, 89 GnuPG, 1117 grep usage, 212, 289, 801, 826, 832, 1046 Hash functions, 526 HASP, 823 Heartbleed, 728, 1019 Heisenbug, 739, 748 Hex-Rays, 123, 219, 330, 717, 1178 Hiew, 105, 152, 812, 817, 887, 888, 892, 920, 1160 Honeywell 6070, 493 IDA, 99, 414, 590, 796, 814, 920, 922, 1143, 1227 var_?, 66, 86 IEEE 754, 243, 346, 410, 469, 1194 Inline code, 214, 343, 582, 634, 664 Integer overflow, 119 Intel 8080, 231 8086, 231, 343, 974 Memory model, 757, 1173 8253, 1068 80286, 974, 1173 80386, 343, 1173 80486, 243 FPU, 243 Intel 4004, 492 Intel C++, 17, 446, 1166, 1175, 1203 Intel syntax, 19, 26 iPod/iPhone/iPad, 26 Itanium, 1169 JAD, Java, 494, 763 John Carmack, 605 Jorge Luis Borges, 852 JPEG, 1115 jumptable, 187, 195 Keil, 26 kernel panic, 873 kernel space, 873 LAPACK, 30 LD_PRELOAD, 878 Linker, 94, 628 Linux, 336, 876, 1050 libc.so.6, 335, 429 Linux kernel, 851 LISP, vii, 704 LLDB, 922 LLVM, 26 long double, 243 Loop unwinding, 207 Lurkmore, 851 LZMA, 1110 Mac OS Classic, 957 Mac OS X, 922 Mathematica, 680, 952 MD5, 526, 822 1240 INDEX memfrob(), 1011 MFC, 889, 1036 Microsoft Word, 728 MIDI, 822 MinGW, 803 minifloat, 485 MIPS, 5, 836, 848, 887, 957, 1114 Branch delay slot, 13 Global Pointer, 32, 326 Instructions ADD, 119 ADD.D, 255 ADDIU, 33, 97, 98 ADDU, 119 AND, 345 BC1F, 292 BC1T, 292 BEQ, 109, 157 BLTZ, 162 BNE, 157 BNEZ, 197 BREAK, 572 C.LT.D, 292 DIV.D, 255 J, 10, 13, 33 JAL, 120 JALR, 33, 120 JR, 186 LB, 218 LBU, 218 LI, 488 LUI, 33, 97, 98, 255, 348, 488 LW, 33, 87, 98, 186, 488 LWC1, 255 MFC1, 258 MFHI, 119, 572, 1220 MFLO, 119, 572, 1220 MTC1, 416 MUL.D, 255 MULT, 119 NOR, 233 OR, 35 ORI, 345, 488 SB, 218 SLL, 197, 237, 364 SLLV, 364 SLT, 157 SLTIU, 197 SLTU, 157, 159, 197 SRL, 241 SUBU, 162 SW, 72 Load delay slot, 186 O32, 72, 76, 77, 1220 Pseudoinstructions B, 216 BEQZ, 159 L.D, 255 LA, 35 LI, 13 MOVE, 33, 96 NEGU, 162 NOP, 35, 96 NOT, 233 Registers FCCR, 291 HI, 572 LO, 572 MS-DOS, 43, 310, 709, 754, 822, 832, 842, 885, 974, 1064, 1067, 1121, 1160, 1173, 1194, 1204, 1208, 1209 DOS extenders, 1173 MSVC, 1225, 1227 Name mangling, 628 Native API, 886 NEC V20, 1069 Non-a-numbers (NaNs), 282 Notepad, 1111 objdump, 414, 878, 892, 920 octet, 493 OEP, 885, 892 OllyDbg, 55, 82, 91, 112, 127, 146, 189, 209, 226, 246, 261, 272, 296, 303, 306, 321, 353, 378, 395, 396, 401, 404, 424, 888, 922, 1227 OOP Polymorphism, 628 opaque predicate, 624 OpenMP, 741, 805 OpenSSL, 728, 1019 OpenWatcom, 804, 859 Oracle RDBMS, 17, 445, 817, 895, 1050, 1057, 1059, 1136, 1147, 1166, 1175 Page (memory), 456 Pareidolia, 851 Pascal, 812 PDP-11, 483 PGP, 816 Phrack, 816 Pin, 604 PNG, 1113 position-independent code, 26, 876 PowerPC, 5, 32, 957 Propagating Cipher Block Chaining, 1028 Punched card, 292 puts() instead of printf(), 28, 84, 122, 153 Python, 604, 680 Quake, 605 Quake III Arena, 419 Racket, 1182 rada.re, 20 Radare, 922 radare2, 1116 rafind2, 851, 920 RAID4, 513 RAM, 94 Raspberry Pi, 26 ReactOS, 901 Recursion, 38, 41, 546 Tail recursion, 546 Register allocation, 460 Relocation, 29 Reverse Polish notation, 292 RISC pipeline, 155 ROM, 94 1241 INDEX Rosencrantz & Guildenstern Are Dead, 852 ROT13, 1011 row-major order, 320 RSA, RVA, 885 SAP, 801, 1045 Scheme, 1182 SCO OpenServer, 964 Scratch space, 860 Security cookie, 309, 907 Security through obscurity, 818 SHA1, 526 SHA512, 741 Shadow space, 114, 115, 470 Shellcode, 623, 873, 886, 1065, 1212 Signed numbers, 144, 499 SIMD, 469, 589 SQLite, 714 SSE, 469 SSE2, 469 Stack, 40, 111, 175 Stack frame, 80 Stack overflow, 41 stdcall, 857, 1160 strace, 878, 922 strtoll(), 1025 Stuxnet, 823 Syntactic Sugar, 174 syscall, 335, 873, 922 Sysinternals, 817, 923 UTF-16LE, 812, 813 UTF-8, 812, 813, 1118 Uuencode, 1118 Uuencoding, 816 VA, 885 Valgrind, 748 Variance, 1009 Watcom, 804 win32 FindResource(), 704 GetOpenFileName, 234 GetProcAddress(), 714 HINSTANCE, 714 HMODULE, 714 LoadLibrary(), 714 MAKEINTRESOURCE(), 704 WinDbg, 922 Windows, 916 API, 1194 IAT, 885 INT, 885 KERNEL32.DLL, 334 MSVCR80.DLL, 422 NTAPI, 931 ntoskrnl.exe, 1050 PDB, 801, 887, 931, 941, 1045 Structured Exception Handling, 46, 893 TIB, 310, 893, 1197 Win32, 334, 813, 878, 885, 1173 GetProcAddress, 891 Tabulation hashing, 514 LoadLibrary, 891 Tagged pointers, 704 MulDiv(), 500, 951 TCP/IP, 520 Ordinal, 889 thiscall, 628, 630, 859 RaiseException(), 893 Thumb-2 mode, 29 SetUnhandledExceptionFilter(), 895 thunk-functions, 30, 891, 958, 966 Windows 2000, 886 TLS, 310, 867, 887, 892, 1197 Windows 3.x, 750, 1173 Callbacks, 870, 892 Windows NT4, 886 Tor, 816 Windows Vista, 885, 931 tracer, 210, 426, 428, 810, 826, 832, 903, 912, 922, Windows XP, 886, 892, 941 1012, 1047, 1054, 1058–1060, 1160 Wine, 901 Turbo C++, 709 Wolfram Mathematica, 1087 uClibc, 730 UCS-2, 494 UFS2, 823 Unicode, 812 UNIX chmod, fork, 732 getopt, 1025 grep, 817, 1162 mmap(), 708 od, 920 strings, 816, 920 xxd, 920, 1093 Unrolled loop, 214, 312, 565, 567, 586 uptime, 878 UPX, 1117 USB, 959 UseNet, 816 user space, 873 UTF-16, 494 x86 AVX, 445 Flags CF, 43, 1202, 1205, 1206, 1209, 1210 DF, 1206, 1210 IF, 1206, 1210 FPU, 1198 Instructions AAA, 1213 AAS, 1213 ADC, 436, 754, 1202 ADD, 17, 53, 111, 575, 754, 1202 ADDSD, 469 ADDSS, 481 ADRcc, 163 AESDEC, 1012 AESENC, 1012 AESKEYGENASSIST, 1015 AND, 18, 334, 335, 338, 352, 365, 403, 1202, 1206 1242 INDEX BSF, 457, 1206 BSR, 1206 BSWAP, 520, 1206 BT, 1206 BTC, 347, 1206 BTR, 347, 917, 1206 BTS, 347, 1206 CALL, 17, 41, 845, 890, 1028, 1107, 1202 CBW, 499, 1206 CDQ, 443, 499, 1206 CDQE, 499, 1206 CLD, 1206 CLI, 1206 CMC, 1206 CMOVcc, 156, 163, 165, 167, 171, 524, 1206 CMP, 99, 1202, 1213 CMPSB, 822, 1206 CMPSD, 1206 CMPSQ, 1206 CMPSW, 1206 COMISD, 477 COMISS, 481 CPUID, 401, 1208 CWD, 499, 754, 1078, 1206 CWDE, 499, 1206 DEC, 225, 1202, 1213 DIV, 499, 1208 DIVSD, 469, 827 FABS, 1210 FADD, 1210 FADDP, 245, 251, 1210 FATRET, 360, 361 FCHS, 1210 FCMOVcc, 284 FCOM, 271, 282, 1210 FCOMP, 259, 1210 FCOMPP, 1210 FDIV, 244, 826, 1211 FDIVP, 245, 1211 FDIVR, 251, 1211 FDIVRP, 1211 FDUP, 785 FILD, 1211 FIST, 1211 FISTP, 1211 FLD, 256, 259, 1211 FLD1, 1211 FLDCW, 1211 FLDZ, 1211 FMUL, 245, 1211 FMULP, 1211 FNSTCW, 1211 FNSTSW, 259, 282, 1211 FSCALE, 418 FSINCOS, 1211 FSQRT, 1211 FST, 1211 FSTCW, 1211 FSTP, 256, 1211 FSTSW, 1211 FSUB, 1211 FSUBP, 1211 FSUBR, 1211 FSUBRP, 1211 1243 FUCOM, 282, 1211 FUCOMI, 284 FUCOMP, 1211 FUCOMPP, 282, 1211 FWAIT, 243 FXCH, 1166, 1211 IDIV, 499, 569, 1208 IMUL, 111, 328, 499, 500, 704, 1202, 1213 IN, 845, 974, 1068, 1209 INC, 225, 1160, 1202, 1213 INT, 43, 1064, 1208 INT3, 810 IRET, 1208, 1209 JA, 144, 283, 499, 1202, 1213 JAE, 144, 1202, 1213 JB, 144, 499, 1202, 1213 JBE, 144, 1202, 1213 JC, 1202 Jcc, 109, 166 JCXZ, 1202 JE, 174, 1202, 1213 JECXZ, 1202 JG, 144, 499, 1202 JGE, 144, 1202 JL, 144, 499, 1202 JLE, 144, 1202 JMP, 41, 65, 891, 1160, 1202 JNA, 1202 JNAE, 1202 JNB, 1202 JNBE, 283, 1202 JNC, 1202 JNE, 99, 144, 1202, 1213 JNG, 1202 JNGE, 1202 JNL, 1202 JNLE, 1202 JNO, 1202, 1213 JNS, 1202, 1213 JNZ, 1202 JO, 1202, 1213 JP, 260, 1069, 1202, 1213 JPO, 1202 JRCXZ, 1202 JS, 1202, 1213 JZ, 108, 174, 1166, 1202 LAHF, 1203 LEA, 81, 114, 383, 534, 551, 575, 861, 934, 1028, 1203 LEAVE, 18, 1203 LES, 979, 1077 LOCK, 916 LODSB, 1068 LOOP, 205, 221, 829, 1077, 1209 MAXSD, 477 MOV, 13, 17, 19, 585, 586, 845, 888, 1028, 1107, 1160, 1204 MOVDQA, 448 MOVDQU, 448 MOVSB, 1204 MOVSD, 476, 587, 995, 1204 MOVSDX, 476 MOVSQ, 1204 MOVSS, 481 INDEX MOVSW, 1204 MOVSX, 223, 231, 395–397, 499, 1204 MOVSXD, 313 MOVZX, 224, 380, 957, 1204 MUL, 499, 500, 704, 1204 MULSD, 469 NEG, 579, 1204 NOP, 551, 1160, 1204, 1213 NOT, 230, 231, 1000, 1204 OR, 338, 608, 1204 OUT, 845, 974, 1209 PADDD, 448 PCMPEQB, 457 PLMULHW, 446 PLMULLD, 446 PMOVMSKB, 457 POP, 17, 40, 41, 1204, 1213 POPA, 1209, 1213 POPCNT, 1209 POPF, 1068, 1209 PUSH, 17, 18, 40, 41, 80, 845, 1028, 1107, 1204, 1213 PUSHA, 1209, 1213 PUSHF, 1209 PXOR, 457 RCL, 829, 1209 RCR, 1209 RET, 10, 13, 17, 41, 309, 630, 751, 1160, 1204 ROL, 361, 1164, 1209 ROR, 1164, 1209 SAHF, 282, 1204 SAL, 1210 SALC, 1069 SAR, 365, 499, 596, 1077, 1210 SBB, 436, 1205 SCASB, 1068, 1069, 1205 SCASD, 1205 SCASQ, 1205 SCASW, 1205 SET, 529 SETALC, 1069 SETcc, 157, 224, 283, 1210 SHL, 236, 295, 365, 1205 SHR, 241, 365, 403, 1205 SHRD, 442, 1205 STC, 1210 STD, 1210 STI, 1210 STOSB, 567, 1205 STOSD, 1205 STOSQ, 586, 1205 STOSW, 1205 SUB, 17, 18, 99, 174, 575, 1202, 1206 SYSCALL, 1208, 1210 SYSENTER, 874, 1208, 1210 TEST, 223, 334, 337, 365, 1206 UD2, 1210 XADD, 917 XCHG, 1204, 1210 XOR, 17, 99, 230, 596, 829, 969, 1160, 1206, 1213 MMX, 445 Prefixes LOCK, 917, 1201 REP, 1201, 1204, 1205 REPE/REPNE, 1201 REPNE, 1205 Registers AF, 492 AH, 1203, 1204 CS, 1173 DF, 730 DR6, 1200 DR7, 1200 DS, 1173 EAX, 99, 122 EBP, 80, 111 ECX, 628 ES, 1077, 1173 ESP, 53, 80 Flags, 99, 146, 1197 FS, 868 GS, 310, 868, 871 JMP, 193 RIP, 877 SS, 1173 ZF, 99, 335 SSE, 445 SSE2, 445 x86-64, 21, 22, 61, 79, 85, 106, 113, 460, 469, 846, 859, 877, 1194, 1200 Xcode, 26 XML, 816, 1009 XOR, 1014 Z80, 493 zlib, 732, 1011 Zobrist hashing, 514 ZX Spectrum, 509 1244 ... IDA listing f1 proc near s = dword ptr -1Ch f1 sub mov call add retn endp f2 proc near s = dword ptr -1Ch f2 sub mov call add retn endp aHello s db 'hello ' db 'world',0xa,0 esp, 1Ch [esp+1Ch+s],... 31 E8 31 48 C3 83 E8 C0 D8 C0 83 EC 08 05 40 00 FE FF FF C4 08 main sub mov xor call xor add retn main proc near rsp, edi, offset format ; "hello, world " eax, eax _printf eax, eax rsp, endp... Intel-syntax by applying the options -S -masm=intel 17 CPU 17 1.5 HELLO, WORLD! call mov leave retn endp main _printf eax, The result is almost the same The address of the hello, world string (stored