1-1. Background: GPU and CPU#

The GPU is a separate physical device connected to the CPU over PCIe. CPU has its own DRAM, GPU has its own VRAM, and data moves between them via DMA over PCIe.

The driver talks to the GPU in two ways:

• MMIO (control plane): CPU reads/writes GPU registers mapped at BAR0 to send commands and config. No bulk data moves here—just signals.
• DMA (data plane): GPU becomes the PCIe Bus Master and transfers data directly between Host Memory ↔ Device Memory, no CPU involvement.

The directions are opposite, which is why they’re separated: MMIO is CPU → device, DMA is device → RAM.

1-2. Implementation in QEMU#

Reference: PCI vs PCIe - QEMU Wiki

Legacy PCI was a shared bus architecture consisting of CPU, Bridge/Memory Controller, Bus, and devices connected to it. The Host Bridge (northbridge) connected CPU/DMA to the bus, and bus arbitration determined bus usage rights.

Reference: QEMU PCI Device Implementation - owalle

This challenge uses -machine q35 to emulate a PCIe-based platform. In PCIe, the Root Complex integrates the roles of the traditional Host Bridge and memory controller, eliminating the need for a separate northbridge chip. Instead of a shared bus, each device connects to the Root Complex via point-to-point links.

Device list confirmed via /sys/bus/pci/devices:

1
[0000:00] pcie.0 (Root Complex = q35 MCH)
2
    ├── 00:00.0  Intel 82G33 Host Bridge    vendor=8086:29c0  class=0x060000
3
    ├── 00:01.0  phantom-gpu (Accelerator)  vendor=1de5:7001  class=0x0b4000  ← DRIVER=gpuctl
4
    ├── 00:1f.0  ICH9 LPC Controller        vendor=8086:2918  class=0x060100
5
    ├── 00:1f.2  ICH9 SATA Controller       vendor=8086:2922  class=0x010601
6
    └── 00:1f.3  ICH9 SMBus Controller      vendor=8086:2930  class=0x0c0500

Since there are no separate bridges or switches in the bus structure, the challenge environment is a single PCIe topology with direct connections from Root Complex to Endpoint, as shown in the [PCI Express Topology] diagram. The phantom-gpu can be seen as the device corresponding to the Graphics position in the [PCI System Block Diagram].

Meanwhile, phantom-gpu has an MMIO region mapped through BAR0:

1
BAR0: 0xfebfe000 ~ 0xfebfefff (4KB MMIO region)

This represents the memory space for register access to the device.

Reference: Getting Started with QEMU - velog

QEMU Process Memory:

A real GPU is an external physical device on PCIe, but phantom-gpu is just a virtual device implemented in C inside the QEMU process. What would be physically separate VRAM is really just a chunk of Guest RAM here, accessed through dma_memory_read/write().

QEMU Abstraction Layer:

QEMU Internal Modules:

• TCG: Emulation engine that JIT-translates Guest x86_64 instructions for execution on the Host CPU
• MMU: Guest virtual address → physical address translation. Dispatches to MemoryRegion callback on BAR0 MMIO access
• QDEV: PCI device emulation framework. phantom-gpu is registered here

The challenge runs in pure TCG mode (no enable-kvm). Whenever an MMIO address (BAR0 region) gets touched during execution, the MemoryRegion callback mapped to that address fires and we land in the phantom-gpu handler.

For reference, the official QEMU PCI device implementation example can be found at hw/misc/edu.c.

phantom-gpu follows similar patterns to the official example—BAR0 MemoryRegion registration, DMA access, interrupt handling—with a shader VM added on top.

1-3. phantom-gpu vs. Real GPU#

Real GPU Compute Cores are parallel hardware, but the phantom-gpu VM executor is just a sequential software interpreter that reads and runs instructions one at a time. No separate MMU, no memory isolation. It decodes the firmware blob, sets up S-box, constants, key material, etc. in the device’s internal struct, and that becomes all the global state the VM needs.

4-1. From init_module to probe#

unk_11A0 is the pci_driver struct. Looking at that address in IDA:

1
__int64 init_module() {
2
    return pci_register_driver(&pci_driver_struct, &_this_module, "gpuctl");
3
}

1
.data:11B0  dq offset "gpuctl"          → .name
2
.data:11B8  dq offset id_table          → .id_table  (vendor=1DE5, device=7001)
3
.data:11C0  dq offset sub_650           → .probe
4
.data:11C8  dq offset sub_30            → .remove

When pci_register_driver registers this struct with the kernel, the kernel scans the PCI bus and automatically calls .probe = sub_650 when it finds a device with vendor=1DE5, device=7001 (phantom-gpu).

4-2. probe(sub_650) — PCI Initialization and DMA Allocation#

1. PCI Activation and BAR0 Mapping

   1
   pci_enable_device(pdev);
   2
   pci_request_regions(pdev, "gpuctl");
   3
   gd->bar0 = pci_iomap(pdev, 0, 4096);
   4
   
   5
   // Signature verification
   6
   sig = ioread32(gd->bar0 + 0x00);
   7
   // sig == 0x5048544D ("PHTM") required
   

   pci_iomap maps the BAR0 physical address to kernel virtual address. After this, reading/writing to this address via ioread32/iowrite32 triggers MMIO access, which triggers QEMU’s MemoryRegion callback.

2. DMA Buffer Allocation and Address Delivery

   For a PCIe device to directly access host RAM, the OS needs to allocate contiguous physical memory first, then tell the device that address. Writing the address to BAR0 doesn’t kick off any transfer—it just tells the device “read from here later.”

   1
   // Request DMA buffer allocation from OS
   2
   gd->cmd_ring = dma_alloc_attrs(&pdev->dev, 0x1000, &gd->cmd_ring_dma, ...);
   3
   gd->log_ring = dma_alloc_attrs(&pdev->dev, 0x1000, &gd->log_ring_dma, ...);
   4
   gd->shader   = dma_alloc_attrs(&pdev->dev, 0x2000, &gd->shader_dma,   ...);
   5
   for (i = 0; i < 4; i++)
   6
       gd->slot[i] = dma_alloc_attrs(&pdev->dev, 0x4000, &gd->slot_dma[i], ...);
   

   The third argument to dma_alloc_attrs (&gd->cmd_ring_dma) is the pointer that receives the physical address. The device directly accesses guest RAM using this physical address.

   Next, these addresses are written to BAR0 registers to inform phantom-gpu:

   1
   // Write DMA physical addresses → BAR0
   2
   iowrite32(lower_32_bits(gd->cmd_ring_dma), gd->bar0 + 0x20);  // cmd_ring lo
   3
   iowrite32(upper_32_bits(gd->cmd_ring_dma), gd->bar0 + 0x24);  // cmd_ring hi
   4
   iowrite32(lower_32_bits(gd->log_ring_dma), gd->bar0 + 0x40);
   5
   iowrite32(upper_32_bits(gd->log_ring_dma), gd->bar0 + 0x44);
   6
   // slots 0~3 similarly written to 0x50~0x6C
   7
   iowrite32(0, gd->bar0 + 0x30);  // doorbell initialization
   

   This is the PCIe DMA Source Address programming stage. phantom-gpu stores these addresses internally and directly reads from those guest memory regions when commands arrive.

3. IRQ Registration and Firmware Init

   1
   // Register MSI interrupt handler
   2
   request_threaded_irq(irq, sub_5E0, NULL, 0x80, "gpuctl", gd);
   3
   
   4
   // Firmware init trigger
   5
   iowrite32(1, gd->bar0 + 0x100);
   6
   _mm_mfence();
   7
   
   8
   // Poll until QEMU completes firmware blob decoding
   9
   for (i = 0; i < 100; i++) {
   10
       if (ioread32(gd->bar0 + 0x104) == 1) break;
   11
       udelay(4295);
   12
   }
   13
   
   14
   pci_set_master(pdev);          // device can be DMA master
   15
   cdev_add(&gd->cdev, ...);      // create /dev/gpuctl
   

   Writing 1 to bar0+0x100 is the firmware init doorbell. QEMU receives this write, decodes the internal firmware blob, sets up gate constants, then changes bar0+0x104 to 1 to signal completion.

4-3. ioctl handler (sub_370) - Command Delivery#

When gpuctl-cli sends an ioctl to /dev/gpuctl, this function receives it. sub_370 is the ioctl handler because it’s in the .unlocked_ioctl field of the file ops registered via cdev_init(gd->cdev, &gpuctl_fops) in probe.

This function handles three commands:

1
mutex_lock(&gd->lock);
2
switch (cmd) {
3

4
case 0x40105001:  // UPLOAD_SHADER
5
    copy_from_user(gd->shader, user_ptr, size);   // user → DMA buffer
6
    push_cmd(gd, CMD_UPLOAD, size, shader_dma);   // write to cmd ring + doorbell
7
    wait_event(gd->wq, gd->irq_done);             // wait for IRQ
8
    break;
9

10
case 0x40105002:  // SET_BUF
11
    copy_from_user(gd->slot[slot], user_ptr, size);
12
    push_cmd(gd, CMD_SETBUF, slot|(size<<32), slot_dma[slot]);
13
    wait_event(gd->wq, gd->irq_done);
14
    break;
15

16
case 0xC0185003:  // DISPATCH
17
    push_cmd(gd, CMD_DISPATCH, max_steps, 0);
18
    wait_event(gd->wq, gd->irq_done);
19
    copy_to_user(user_out, gd->slot[2], min(out_size, 0x4000));  // return result
20
    break;
21
}
22
mutex_unlock(&gd->lock);

The role of push_cmd(sub_160) is important here:

1
// 1. Write 64-byte command to next slot in cmd ring (writes to DMA buffer)
2
memcpy(&cmd_ring[tail], &cmd, 64);
3

4
// 2. doorbell write → triggers QEMU MMIO write handler
5
iowrite32(next_idx, gd->bar0 + 0x30);

cmd_ring operates as a streaming DMA ring buffer pattern. The driver advances the tail (Producer index) while filling commands, and phantom-gpu consumes from the head (Consumer index).

Fill the DMA buffer first, then wake the device with a doorbell—that’s standard PCIe DMA ordering, and iowrite32(next_idx, bar0+0x30) is that signal. After the doorbell write, phantom-gpu reads cmd_ring via DMA and processes the command.

4-5. IRQ handler (sub_5E0) - Completion Signal#

1
irqreturn_t irq_handler(int irq, void *gd) {
2
    u32 status = ioread32(gd->bar0 + 0x14);   // check completion status
3
    if (status) {
4
        gd->log_record = *(u64 *)gd->log_ring; // read log
5
        gd->has_log = 1;
6
        iowrite32(status, gd->bar0 + 0x18);    // IRQ ACK
7
        wake_up(&gd->wq);                       // wake ioctl handler
8
    }
9
    return IRQ_HANDLED;
10
}

After DISPATCH completes, phantom-gpu fires an MSI and this ISR wakes up. It unblocks the ioctl handler that’s been sitting on wait_event, and the result (slot2 = flag output area) gets sent back to userspace via copy_to_user.

4-6. Complete Flow#

1
gpuctl-cli
2
    │ ioctl(fd, DISPATCH, ...)
3
    ▼
4
sub_370 (ioctl handler)
5
    ├─ copy_from_user → DMA buffer (slot[0], slot[1])
6
    ├─ push_cmd
7
    │     ├─ write command to cmd_ring[tail]  ← streaming ring buffer
8
    │     └─ iowrite32(next_idx, bar0+0x30)  ← doorbell
9
    │                   │
10
    │        Real HW:   └─ PCIe TLP → device DMA engine → read host RAM
11
    │        QEMU:      └─ MMIO trap → MemoryRegion callback
12
    │                        └─ dma_memory_read(cmd_ring_dma)  ← direct guest RAM
13
    │                                    │
14
    │                      shader validation + VM execution
15
    │                      MSI interrupt generated
16
    │                   │
17
    └─ wait_event ◀─────┘
18
                        │
19
                    sub_5E0 (IRQ handler)
20
                        ioread32(bar0+0x14)   completion status
21
                        iowrite32(bar0+0x18)  ACK
22
                        wake_up(&gd->wq)
23
                        │
24
    ioctl handler resumes ◀┘
25
    copy_to_user(slot[2]) → return flag output

As mentioned, gpuctl.ko is just a forwarding layer—copy to DMA buffer, ring the doorbell, done. The real analysis target is the qemu-system-x86_64 binary.

5-1. Firmware Blob Extraction and Decoding#

In sub_428AB0, the offset == 0x100 branch is the firmware init path, referencing the embedded blob at 0xB0A2D8.

The decoding chain is straightforward:

1
; ① xorshift32 — 0x428FE8
2
mov  edx, 0FE170E07h          ; seed
3
loc_428FE8:
4
    mov  eax, edx
5
    shl  eax, 0Dh             ; state ^= state << 13
6
    xor  eax, edx
7
    mov  edx, eax
8
    shr  edx, 11h             ; state ^= state >> 17
9
    xor  eax, edx
10
    mov  edx, eax
11
    shl  edx, 5               ; state ^= state << 5
12
    xor  edx, eax
13
    xor  [rcx], dl             ; blob[i] ^= low_byte(state)
14
    add  rcx, 1
15
    cmp  rcx, rbp              ; 0x2E1 bytes
16
    jnz  loc_428FE8
17

18
; ② CRC32 — 0x42901C
19
mov  eax, 0FFFFFFFFh           ; CRC init
20
loc_429028:
21
    movzx edx, byte ptr [rsi]
22
    xor   edx, eax
23
    shr   eax, 8
24
    movzx edx, dl
25
    xor   eax, [rcx+rdx*4]    ; table @ 0x19EEF00
26
    cmp   rsi, rbp
27
    jnz   loc_429028
28
cmp  eax, 0AEF1393Ah          ; expected CRC
29

30
; ③ LZ4 raw block — 0x429080
31
movzx ebx, byte ptr [rdx]     ; token byte
32
    shr  al, 4                 ; literal_length = token >> 4
33
    cmp  al, 0Fh               ; if 0xF → extended length loop
34
    jz   loc_429522

Here’s the extraction script.

1
#!/usr/bin/env python3
2
import struct, zlib, sys, lz4.block
3
from pathlib import Path
4

5
qemu = Path("qemu-system-x86_64").read_bytes()
6
enc = bytearray(qemu[0xB0A2D8 : 0xB0A2D8 + 0x2E1])
7

8
# xorshift32 decrypt (seed=0xFE170E07)
9
state = 0xFE170E07
10
for i in range(len(enc)):
11
    state ^= (state << 13) & 0xFFFFFFFF
12
    state ^= state >> 17
13
    state ^= (state << 5) & 0xFFFFFFFF
14
    state &= 0xFFFFFFFF
15
    enc[i] ^= (state & 0xFF)
16

17
# CRC32 verify — raw accumulator (no final XOR) == 0xAEF1393A
18
crc_raw = zlib.crc32(bytes(enc)) ^ 0xFFFFFFFF & 0xFFFFFFFF
19
assert crc_raw == 0xAEF1393A, f"CRC mismatch: {crc_raw:#x} vs 0xAEF1393A"
20

21
# LZ4 raw block decompress → 0x308 bytes
22
firmware = lz4.block.decompress(bytes(enc), uncompressed_size=0x308)
23
Path("firmware_decompressed.bin").write_bytes(firmware)
24
print(f"OK: {len(firmware)} bytes → firmware_decompressed.bin")

5-2. Firmware → Device State Copy#

The 0x308-byte blob after LZ4 decompression is copied to various offsets of the device state struct. Reconstructing the structure from mov/rep movsq assembly traced in IDA:

1
  typedef struct PhantomGPUState {
2
  uint8_t  _pci_header[0xC00];
3

4
  uint8_t  key_seed[8];
5
  uint8_t  blob_copy[0x308];
6

7
  uint64_t blob_tail_qword;
8

9
  uint8_t  trace_pattern[8];
10
  uint32_t gate_reg7;
11
  uint32_t gate_buf2_dw0;
12
  uint32_t gate_crc_buf1;
13
  uint32_t gate_xor_r0r3;
14

15
  uint32_t post_f28;
16
  uint32_t post_f2c;
17

18
  uint64_t dispatch_table[256];
19

20
  uint8_t  _dma_addrs[];
21

22
  uint32_t reg[16];
23
  uint32_t pc;
24
  uint32_t sp;
25
  uint32_t cmp_flag;
26

27
  uint8_t  active_shader[0x2000];
28

29
  uint8_t  trace_ring[16];
30
  uint32_t trace_count;
31

32
  uint32_t gate_armed;
33
  uint32_t vm_status;
34
  uint32_t fault_code;
35

36
} PhantomGPUState;

5-3. Phase 1 Transform — Grammar A / Grammar B#

I’ll call the firmware blob → gate constant derivation process “Phase 1 Transform.” In the IDA range 0x429240~0x429472, two distinct patterns repeat.

Grammar A (stream XOR) — trace pattern generation

0x429240~0x429261 loop:

1
loc_429240:
2
    lea  edx, [ecx+eax]         ; edx = (seed + i) — seed is low byte of blob[0x08]
3
    movzx edx, dl               ; dl = (seed + i) & 0xFF
4
    movzx edx, byte ptr [rdi+rdx+0D0Ch]  ; dev[0xD0C + dl] = sbox[dl]
5
    xor   dl, [rax+0C07h]       ; ^ blob[i-1] (dev[0xC08+i-1])
6
    mov   [rax+0F0Fh], dl       ; → dev[0xF10+i-1] (trace pattern)
7
    add   rax, 1
8
    cmp   rax, rsi              ; 8 iterations
9
    jnz   loc_429240

Pseudo-code:

1
// Grammar A: sbox byte XOR
2
for (i = 0; i < 8; i++)
3

4
dev[0xF10 + i] = sbox[(seed + i) & 0xFF] ^ blob[i];

seed is the low byte of blob[0x08:0x0C] (= dev[0xC10]), sbox is dev[0xD0C] = blob[0xFC:0x1FC]. The result is 50 11 21 40 20 11 51 41 — the expected trace value for gate bit 0.

Grammar B (pack_perm XOR) — 32-bit constant generation

0x42926B~0x4292C1 is the first Grammar B block. Repeating pattern:

1
mov  edx, [rbx+0C18h]       ; edx = selector (4-byte value in blob)
2
lea  eax, [rdx+1]           ; idx+1
3
lea  ecx, [rdx+2]           ; idx+2
4
movzx eax, byte ptr [rbx+rax+0D0Ch]  ; sbox[(selector+1) & 0xFF]
5
movzx ecx, byte ptr [rbx+rcx+0D0Ch]  ; sbox[(selector+2) & 0xFF]
6
shl  eax, 8                 ; byte1 << 8
7
shl  ecx, 10h               ; byte2 << 16
8
or   eax, ecx
9
movzx ecx, dl               ; sbox[(selector+0) & 0xFF]
10
movzx ecx, byte ptr [rbx+rcx+0D0Ch]
11
add  edx, 3
12
movzx edx, byte ptr [rbx+rdx+0D0Ch]  ; sbox[(selector+3) & 0xFF]
13
or   eax, ecx               ; | byte0
14
shl  edx, 18h               ; byte3 << 24
15
or   eax, edx               ; = pack_perm result
16
xor  eax, [rbx+0C14h]       ; ^ XOR source from blob
17
mov  [rbx+0F18h], eax       ; → dev[0xF18] (gate_reg7)

Pseudo-code:

1
// Grammar B: 4-byte S-box lookup → u32 → XOR
2
uint32_t pack_perm(uint8_t *sbox, uint32_t idx) {
3
    return sbox[(idx+0) & 0xFF]        |
4
          (sbox[(idx+1) & 0xFF] << 8)  |
5
          (sbox[(idx+2) & 0xFF] << 16) |
6
          (sbox[(idx+3) & 0xFF] << 24);
7
}
8

9
dev[0xF18] = pack_perm(sbox, blob_dword(selector_offset)) ^ blob_dword(xor_offset);

This pattern repeats 6 times to generate 6 gate constants:

1
// (selector offset → XOR source → storage location) mapping traced from IDA:
2
dev[0xF18] = pack_perm(sbox, blob[0x10]) ^ blob[0x0C]   // gate_reg7    = 0xA5A5A5A5
3
dev[0xF1C] = pack_perm(sbox, blob[0x18]) ^ blob[0x14]   // gate_buf2    = 0xC0DEF00D
4
dev[0xF20] = pack_perm(sbox, blob[0x20]) ^ blob[0x1C]   // gate_crc     = 0x2A8DED84
5
dev[0xF24] = pack_perm(sbox, blob[0x58]) ^ blob[0x54]   // gate_xor     = 0xCC99E897
6
dev[0xF28] = pack_perm(sbox, blob[0x28]) ^ blob[0x24]   // post_f28     = 0x13579BDF
7
dev[0xF2C] = pack_perm(sbox, blob[0x30]) ^ blob[0x2C]   // post_f2c     = 0x2468ACE0

Most are sequentially arranged at 8-byte intervals from blob[0x0C], but f24 (gate_xor) is separated at blob[0x54]/blob[0x58].

5-4. Dispatch Table Initialization#

The dispatch table is initialized immediately after Phase 1 Transform. 15 (opcode, handler) pairs are statically defined at unk_1719400, and the firmware blob’s scramble value determines the final slot position:

1
// 0x4294C2 — dispatch table initialization loop
2
for (int i = 0; i < 15; i++) {
3
    uint8_t  opcode  = static_table[i].opcode;      // unk_1719400
4
    void    *handler = static_table[i].handler;
5
    uint8_t  slot    = (13 * opcode + scramble_data[opcode] + blob[0x304]) & 0xFF;
6
    dev->dispatch_table[slot] = handler;
7
}
8
// scramble_data = blob[0x1FC:0x2FC], blob[0x304] = global constant

Without this scramble you can’t figure out the opcode → handler mapping, so firmware decoding is a hard prerequisite for VM analysis.

Recovered ISA — 15 opcodes, fixed 4-byte length (byte0=opcode, byte1=Rd, byte2=Rs/Rb/buf, byte3=imm/offset):

5-5. Shader Validator (sub_428450)#

The uploaded shader gets validated before dispatch:

1
int validate_shader(PhantomGPUState *dev) {
2
    uint8_t *shader = dev->active_shader;
3
    int size = dev->shader_size;
4

5
    // 1. Size/alignment
6
    if (size == 0 || (size & 3) || size > 0x1803)
7
        return FAIL;
8

9
    // 2. Opcode validity — iterate all instructions
10
    for (int pc = 0; pc < size; pc += 4) {
11
        uint8_t op = shader[pc];
12
        if (op > 0x51)                  // switch 82 cases, default → FAIL
13
            return 7;
14
        // jump table: only 0x01,0x02,0x10~0x13,0x20,0x21,0x30~0x32,0x40,0x41,0x50,0x51 allowed
15
        // others → return 7
16
    }
17

18
    // 3. BFS reachability marking (sub_4275B0)
19
    //    Starting from PC=0, follow CALL/BEQ/BNE targets to mark reachable blocks
20
    mark_reachable(shader, size);  // sub_4275B0
21

22
    // 4~5. Prologue/epilogue checks only for reachable blocks
23
    for (each reachable instruction) {
24
        if (op == 0x41) {              // RET
25
            // Two instructions before RET must be POP (0x51)
26
            // Additionally, POP target registers must be r15, r14 in order
27
            word_m2 = read_insn(pc - 8);  // byte[pc-12..pc-9]
28
            word_m1 = read_insn(pc - 4);  // byte[pc-8..pc-5]
29
            if ((word_m2 & 0xFF) != 0x51) return 4;
30
            if ((word_m1 & 0xFF) != 0x51) return 4;
31
            // Check word_m1 byte1 == 0x0E (r14)
32
            // Check word_m2 byte1 == 0x0F (r15)
33
        }
34
        if (op == 0x40) {              // CALL
35
            // First 2 instructions at CALL target must be PUSH r15(0x50 0x0F) + MOV r15,r14(0x01 0x0F 0x0E)
36
            target = pc + offset*4;
37
            word0 = read_insn(target);
38
            word1 = read_insn(target + 4);
39
            if ((word0 & 0xFF) != 0x50) return 4;   // PUSH
40
            if ((word0 >> 8 & 0xF) != 0xF) return 4; // r15
41
            if ((word1 & 0xFF) != 0x01) return 4;   // MOV
42
            if ((word1 >> 8 & 0xF) != 0xF) return 4; // Rd=r15
43
            if ((word1 >> 16 & 0xF) != 0xE) return 4; // Rs=r14
44
        }
45
    }
46
    return 0;  // pass
47
}

Checks 4 and 5 only apply to blocks marked reachable by sub_4275B0 (BFS). Anything BFS can’t reach completely skips prologue/epilogue validation.

This is the key to bypassing gate verification later.

5-6. VM Executor Loop & Fault Codes#

At dispatch start, memset(dev + 0x1730, 0, 0x20a8) initializes all registers/PC/SP/trace ring, then executes the following loop:

1
while (steps < max_steps) {
2
    if (pc >= shader_size) { fault = FAULT_BAD_PC; break; }
3

4
    uint8_t opcode = shader[pc];
5
    uint8_t slot = (13 * opcode + scramble_data[opcode] + blob[0x304]) & 0xFF;
6
    handler_fn *handler = dev->dispatch_table[slot];
7

8
    if (!handler) { fault = FAULT_BAD_OPCODE; break; }
9

10
    trace_ring[trace_count++ % 16] = opcode;
11
    handler(dev, &shader[pc]);
12

13
    if (dev->fault_code) break;
14
    steps++;
15
}

dispatch_table[256] is a function pointer array—only the 15 slots from 5-4 have real handlers, everything else is NULL. An invalid opcode hits a NULL handler → instant FAULT_BAD_OPCODE.

Each handler writes a constant to dev->fault_code (offset 0x379C) on error. Tracking this pattern gives us the fault codes:

Summary:

6-1. Gate Condition Analysis#

The RET handler (sub_428370) calls sub_4280D0 (checker) on every RET. If it returns exactly 0x3F (all 6 bits set), vm_status = 8 (GATE_TRIGGERED).

Bit 0 (0x01) — Trace History

1
Last 8 opcodes == 50 11 21 40 20 11 51 41
2
(PUSH XOR STORE CALL LOAD XOR POP RET)

Bit 1 (0x02) — Register Check

1
reg[7] == 0xA5A5A5A5
2
reg[0] ^ reg[3] == 0xCC99E897

Bit 2 (0x04) — buf2 First Dword

1
*(u32*)buf2 == 0xC0DEF00D

Bit 3 (0x08) — buf1 CRC32

1
CRC32(buf1[0:16]) == 0x2A8DED84

The gate only checks the CRC of buf1’s first 16 bytes—not the actual byte values.

Bit 4 (0x10) — Gate Armed

1
On CALL execution: reg[1] == 0x47415445 ("ETAG") + previous 4 traces == [LOAD, AND, STORE, CALL]

Bit 5 (0x20) — Shader CRC32

1
reg[2] == CRC32(entire shader bytecode)

Dead code bytes are also included in the CRC calculation.

6-2. Gate Verification Bypass#

Problem:

In the Bit 0 history 50 11 21 40 20 11 51 41, right before RET is a single 0x51 (POP), and before that is 0x11 (XOR). But the validator demands two POPs before any reachable RET. You can’t satisfy both constraints with reachable code alone.

Solution:

The validator doesn’t check CALL prologue/RET epilogue rules on dead code blocks. We can exploit that.

1. Within a reachable function, use STORE to overwrite the return address in slot3 (stack) with the dead block PC
2. End the function with POP + POP + RET that satisfies validator rules
3. RET pops the overwritten address and jumps to the dead block
4. In the dead block, create exactly the 50 11 21 40 20 11 51 41 history
5. On the dead block’s final RET, checker is called → 0x3F → GATE_TRIGGERED

Dead Block Return Address Overwrite

gpuctl-cli run allocates slot3 as 16KB (SP=4096). CALL pushes the return address to stack[4095], but STORE’s offset (byte3) is signed 8-bit—with base 0 you can only reach up to stack[15].

The trick: preload r14 = 4080 before CALL. The function prologue’s MOV r15, r14 gives us r15 = 4080, and STORE r9, [buf3, r15+15] lands exactly on stack[4095]. The validator checks the bytecode statically, so it can’t catch the same instructions writing to different locations depending on the runtime value of r14.

6-3. Shader Layout#

The final shader has two regions: a reachable region and a dead block. The reachable region sets gate bits 1~5, then a return address overwrite jumps into the dead block to nail bit 0 (trace history) as well.

7-1. Gate Check Scope#

What the gate does fix:

• trace8 — 8-byte match
• reg7 — value match
• reg0 ⊕ reg3 — XOR result only (individual values are free)
• buf2[0:4] — value match
• CRC32(buf1[:16]) — CRC only (raw bytes are free)
• reg2 — CRC32(shader) match

What the gate does not check:

• Individual reg0, reg3 values
• reg4, reg5
• All 32 bytes of buf1 (first 16: CRC only, last 16: completely free)

7-2. What the Success Path Actually Reads#

1
// sub_428AB0 success path, 0x4299d6..
2
v132 = f28 ^ reg0                                // uses reg0 directly
3
v134 = f2c ^ ((reg4 + reg5) & 0xFFFFFFFF)        // uses reg4, reg5 directly
4
v135 = ROL32(reg3, 7) ^ v132                     // uses reg3 directly
5
msg  = trace8 || pack_le32(v135) || pack_le32(v134) || buf1[16:32]
6
rounds = expand_round_material(buf1[0:16])        // uses raw buf1 bytes directly

So even after passing the gate, you still need to recover the exact individual values of reg0, reg3, reg4, reg5 and the precise 32 bytes of buf1 to get the real flag.

7-4. Success Path Analysis#

1
buf1[0:32]  ──┐
2
trace8      ──┤
3
reg0~reg5   ──┼──▶ compress_states() ──▶ S0 (AES key), S1 (CTR counter)
4
f28, f2c    ──┘
5
                         ↓
6
              AES-128-CTR(S0, S1) ──▶ keystream (78B)
7
                                           │
8
              fixed_blob (78B) ──XOR───────┘──▶ flag (buf2 output)

Main logic of compress_states:

1. Generate 24 round materials from buf1[0:16] (MurmurHash3 fmix variant)
2. GF(2^8) MixColumns (standard + custom matrix) + Feistel 12 rounds
3. fold16 (sub_4279B0): 16B → 64-bit keyed digest (seed 0xDEADBEEFCAFEBABE)
4. Reverse Feistel 8 rounds + final GF MixColumns → S0 (AES key), S1 (CTR counter)

This is a non-invertible one-way function, so reversing it seems impossible. However, since it runs automatically inside the VM with the correct inputs, further analysis isn’t necessary.

7-6. Hidden Witness#

So the flag doesn’t pop out even after passing the gate—we need to find the exact input values the success path expects. Luckily, there’s still an uninterpreted region left in the firmware blob’s 0x308 bytes.

This part takes a bit of guessing:

Hidden Registers

Decoding firmware blob 0x034..0x054 with Grammar B (pack_perm XOR):

Verification: 0x12345678 ^ 0xDEADBEEF = 0xCC99E897 = exactly matches the expected reg0 ^ reg3 value for gate bit 1.

Hidden buf1

Firmware blob blob[0x5C:0x80] region:

1
[0x5C:0x7C] = raw 32 bytes (data encoded with Grammar A)
2
[0x7C:0x80] = selector dword (0x0000009D)

Same [raw data] + [selector dword] format as the Grammar A block used for trace8. Applying Grammar A:

1
decoded[i] = blob[0x5C + i] ^ sbox[(0x9D + i) & 0xFF]

Out comes PHTM_CTF_KEY_128SECONDHALF_DATA!!

The identical [raw data] + [selector dword] format, a meaningful ASCII string, and CRC32 of the first 16 bytes = 0x2A8DED84 = exactly matching gate constant f20—all signs point to this being the intended buf1 value.

Note that hidden reg and buf1 aren’t referenced by the QEMU success path at runtime. These hidden markers are concealed values you need to discover, not answers the VM consumes directly.