A Practical Primer on Malware Analysis
When a suspicious binary lands on your desk, the clock is ticking. Whether it arrived via a phishing email, a quarantined AV alert, or an anomalous process on a server — your goal is the same: understand what it does, fast, with minimum exposure. This post covers the fundamentals every analyst should know: how malware is classified, the techniques it uses, and how to triage an unknown sample with confidence.
1. Malware Classification
Not all malware is created equal. Before diving into analysis, it helps to know what category of threat you’re dealing with — each class has characteristic behaviors and typical goals.
Class
Description
Virus
Propagates across systems but requires user intervention to spread
Worm
Self-propagates across networks with no user interaction required
Bot
Automated process that interacts with other network services — often part of a botnet
Trojan
Disguised as legitimate software. Gets in through trust
Ransomware
Holds victim data hostage using cryptography or other coercive means
Rootkit
Masks its own existence or the presence of other malicious software
Backdoor / RAT
Enables a remote attacker to access or issue commands to a compromised machine
Info Stealer
Silently harvests passwords, credentials, and personal data
Dropper / Downloader
Stage-one payload designed purely to deliver or install other malware
Adware / PUP
Potentially unwanted programs that generate revenue or gather data with minimal consent
HackTool
Offensive tools (e.g. port scanners, exploit frameworks) that aren’t inherently malicious
Hoax
Delivers false virus warnings or installs a fake antivirus to extort the user
2. Malware Techniques
A single piece of malware rarely belongs to just one class — and nearly always employs several techniques to do its job. The MITRE ATT&CK framework catalogs these exhaustively. Here are the most common ones to watch for.
Obfuscation
Making code deliberately hard to read and analyze. Expect base64 or XOR-encoded strings, junk functions that do nothing, control-flow flattening, and string encryption designed to exhaust reverse engineers.
Persistence
Once inside, malware wants to stay. The persistence mechanism it uses is often a reliable fingerprint for identifying a specific malware family across different infections.
Privilege Escalation
Exploiting bugs, design flaws, or configuration mistakes to gain elevated access. Common techniques include DLL search order hijacking, DLL injection, buffer/stack overflows, heap spraying, Return Oriented Programming (ROP), credential theft, and UAC bypasses.
Defense Evasion
Staying off the radar. Malware may kill AV processes, self-delete after execution, use stolen code-signing certificates, or hollow out legitimate processes to run under their cover. Other techniques include timebombs, timestomping, DLL sideloading, masquerading, and code injection.
Credential Theft
Going after password storage, keylogging credentials as they’re typed, and taking screenshots of sensitive activity.
Reconnaissance, Lateral Movement & Execution
Beyond the initial foothold, malware surveys the network and pivots to other systems to maximize reach. This typically involves running scripts or post-exploitation tooling, collecting sensitive files, exfiltrating data, and maintaining command-and-control (C2) communication back to the attacker.
Key distinction: A debugger interacts with a program as it runs, while a disassembler and a decompiler perform static analysis — converting binary code into human-readable form at different levels of abstraction. Knowing when to use which tool is half the battle.
3. Triage Analysis: The ER Approach
Malware analysts borrow the triage mindset from emergency medicine: narrow down the critical information quickly before committing to deep analysis. The goal is to make a confident call on severity and behavior with the least time investment possible.
When you receive an unknown binary, the first question is always: how did it get here? Did it arrive via email? A browser download? Was it flagged by AV? Is it an anomalous running process? Context shapes everything.
Step 1 — File Info & Header Analysis
Run file to determine the file type, then verify the file header in a hex editor like HxD. Malware often lies about its extension.
Step 2 — PE Header Parsing
Use PE Bear to inspect resources, DLL imports, and libraries. Spotting Ws2_32.dll is an immediate flag — it’s the Windows socket library and signals network activity.
Step 3 — Hash Lookup
Calculate the file hash and check it against threat intel databases. If it’s known malware, you may already have a full analysis available.
Step 4 — String Extraction
Use strings on Linux or BinText on Windows. Even partial strings — URLs, registry keys, error messages — can reveal intent quickly.
Step 5 — AV & VirusTotal
Submit the sample to VirusTotal for multi-vendor detections. Even partial hits can give you a family name to research further.
Step 6 — VM Detonation
Use sandboxed services like hybrid-analysis.com to get behavioral output fast — file drops, registry changes, process trees, and more.
Step 7 — Network Capture
Review any network connections, DNS queries, or C2 beaconing from the detonation output. If sandbox data isn’t available, dynamic debugging becomes necessary.
This triage checklist gets you through initial analysis in under an hour for most samples. From here, deeper static analysis (disassembly, decompilation) and dynamic debugging take over — but having solid triage behind you means you already know what questions to ask.