Skip to main content
Hardware-assisted Virtual Machine 
노용환 (a.k.a. somma) 
fixbrain@gmail.com
system utilization 
consolidation 
management cost 
isolation 
trusted environment resource aggregation 
GRID system 
MPP (Massively Parallel Processing) 
resource access control 
mobility 
emulation
1960 1970 1999 2006 현재 
System/370, IBM 
x86 virtualization, VMWare 
CP-40, IBM, 
Cambridge Scientific Center 
full virtualization 
application virtualization 
(application streaming) 
x86,x64, ARM, … 
Storage, 
Network 
… 
VMWare, Virtual Box, Xen… 
… 
OpenStack, CloudStack,… 
… 
Amazon, Google…
Guest OS Guest OS 
Memory and I/O 
Virtualization 
Shared Device 
VMM 
Physical H/W 
Control 
CPU CPU MEMORY 
virtualized h/w 
physical h/w 
VMM must … 
- support same hardware interface 
- can control guest OS when accessing H/W resources.
Types of operation… 
mov eax 
mov ebx 
… 
Direct Execution 
eflags 
control registers 
MSR 
privileged instructions 
????
Full Virtualization 
- No OS modification 
- Emulating, Binary translation, Trace cache,… 
- VMware ESX server 
- QEMU 
Para Virtualization 
- Need OS modification 
- Hypercall 
- Xen 
- Bochs
Hardware Assisted Virtualization 
Virtualize… 
CPU 
- AMD-V , VT-x 
IOMMU 
- AMD-Vi, VT-d 
Network 
- VT-c 
VMX operation 
VMX root operation 
VMX non-root operation
Hardware Assisted Virtualization 
Trap based development for VirtualMachine 
- handle_cupid_instruction() 
- handle_mov_crX() 
- handle_read_msr() 
- handle_write_msr() 
- … 
HW based Hypervisor programming = VMEXIT handler programming
VMX (Intel Virtual Machine Extension) 
VMXON 
VMCLEAR 
VMPTRLD 
VMWRITE 
VMLAUNCH 
GUEST Exit 
VMREAD 
VMRESUME 
VMXOFF
VMX – new instructions, new data structure 
VMXON Region 
- created per logical processor 
- used by VMX instructions 
VMCS Region 
- created per virtual CPU for guest OS 
- used by CPU and VMM 
- 4Kb aligned 
- PHYSICAL_ADDRESS == typedef LARGE_INTEGER 
- …
VMM (Virtual Machine Monitor) programming summary 
check VMX support allocate VMXON region execute VMXON 
execute VMPTRLD execute VMCLEAR allocate VMCS region 
initialize VMCS data 
host-state area fields 
VM-exit control fields 
VM-entry control fields 
VM-execution control fields 
guest-state area fields 
execute VMLAUNCH handling various VM-exits
VMCS data organization 
#1 Guest state fields 
- saved on VM exits, loaded on VM entries 
#2 Host state fields 
- loaded on VM exits 
#3 Execution control fields 
- control VMX-non root operations 
#4 Exit control fields 
- control VM exits 
#5 Entry control fields 
- control VM entries 
#6 VM Exit info 
- saved VM exits information on VM exits 
pin-based controls 
processor-based controls 
exception-bitmap address 
I/O bitmap address 
Timestamp counter offset 
CR0/CR4 guest/host masks 
CR3 targets 
MSR bitmaps
Accessing VMCS data 
VMREAD 
VMWRITE 
virtual address / physical address 
READ 
virtual address / physical address 
WRITE
Accessing VMCS data
Initialize VMM and Run VMM
Handling VM exits 
#6 VM Exit info
Handling VM exits
Virtual Machine Threat
Attacks on Binary Translator 
CVE-2009-1542 - VirtualPC instruction decoding 
• wbinvd (write back and invalidate cache), clts (clear task-switched flag in cr0) 
CVE-2008-4915 - VMware, Trap Flag Set by IRET Not Cleared for CCh Instruction 
CVE-2009-2267 - VMware Mishandled Exception on Page Faults 
… 
Attacks on Para-virtualization 
CVE-2008-4279 - VMware, Interrupt Can Occur at NonCanonical RIP After Indirect Jump 
CVE-2012-0217 - Advanced Exploitation of Xen Hypervisor Sysret VM Escape Vulnerability 
( http://www.vupen.com/blog/20120904.Advanced_Exploitation_of_Xen_Sysret_VM_Escape_CVE-2012-0217.php ) 
… 
Attacks on Device Emulation / Acceleration 
CVE-2012-0217 ( http://www.vupen.com/blog/20120904.Advanced_Exploitation_of_Xen_Sysret_VM_Escape_CVE-2012-0217.php )
Attacks on HVM 
CVE-2009-3827 - Virtual PC VMExit Event Confusion 
• exit reason MOV_CR, MOV_DR 
• MOV_CR : check guest cpl == 0 
• MOV_DR : !! 
• ring3 에서 DR 레지스터를 조작가능 !? DoS ?! 
CVE-2009-3722 - KVM VMExit Event Confusion 
• CVE-2009-3827 와 동일한 버그 
더 자세한 내용은 http://www.cr0.org/paper/jt-to-virtualisation_security.pdf 를 참고하세요.
VM Detection 
너무 많다!
HVM base rootkit 
최초의 가상머신 기반 루트킷 ( http://www.invisiblethingslab.com/resources/bh07/IsGameOver.pdf )
HVM base rootkit – keylogger 
PS/2 
Port 0x60 
Keyboard Controller 
Keyboard 
Mouse 
CPU 
Port 0x64 
CPU 가상화 
HVM rootkit 
• CPU 의 특권 명령을 가로챔 (e.g. IN, OUT) 
• PORT I/O 를 OS 보다 먼저 하드웨어 레벨에서 처리
Attack Hypervisor ?! or Another Attack Surface 
OS / Device Drivers 
Hypervisor 
BIOS 
Chipset 
OS Level 
HVM rootkit 
rootkit code in SMM / ACPI / UEFI / PCI 
CPU CPU bugs ? Micro code update ?
http://leaksource.files.wordpress.com/2013/12/nsa-ant-souffletrough.jpg
감사합니다. 
연락은 fixbrain@gmail.com 으로…