Testing

Testing approach and how to run tests in oboromi

Testing is crucial for ensuring correctness in emulation. This page covers the current test suite and how to run tests.

Running Tests

To run all tests in the project:

cargo test

For verbose output showing test details:

cargo test -- --nocapture

The project is in early development, so test coverage is still being expanded. The current focus is on CPU instruction correctness.

Test Structure

The test suite is organized into several categories:

CPU Instruction Tests

Located in core/src/tests/run.rs, these tests verify ARM64 instruction emulation using the Unicorn engine backend.

Test Framework

The test framework provides:

• Warmup phase - JIT compiler initialization to avoid cold-start timeouts
• Timeout detection - Platform-specific timeouts (100ms Linux, 500ms macOS)
• Result tracking - Pass/fail status with execution duration
• ARM64 instruction helpers - Functions to encode ARM64 instructions

Example: Running a CPU Test

run_test(
    "ADD X1, X1, #2",
    &[arm64::add_imm(1, 1, 2)],
    |cpu| {
        cpu.set_x(1, 5);  // Setup: X1 = 5
    },
    |cpu| cpu.get_x(1) == 7,  // Verify: X1 = 7
)

This test:

1. Encodes the ARM64 instruction ADD X1, X1, #2
2. Sets register X1 to 5
3. Executes the instruction
4. Verifies X1 now contains 7

Current Test Cases

The following ARM64 instructions are tested:

1. Basic Instructions

   • NOP - No operation
   • ADD immediate - ADD X1, X1, #2
   • SUB immediate - SUB X2, X2, #1
   • ADD register - ADD X0, X0, X1
   • MOV register - MOV X3, X4
   • RET - Return instruction
2. Complex Sequences

   • Atomic ADD test - Large immediate values
   • Memory access pattern - Multiple sequential instructions
   • Multiple arithmetic ops - Combined ADD, SUB, and register operations

All tests are executed through the run_tests() function, which can be triggered from the GUI or run via cargo test.

Multicore Tests

Located in core/src/tests/multicore_test.rs, these tests verify the multi-core CPU manager:

test_multicore_initialization

Verifies that the CPU manager correctly initializes:

• 8 CPU cores (matching Switch 2 architecture)
• 12GB of shared memory

Reference: core/src/tests/multicore_test.rs:6

test_shared_memory_access

Verifies that multiple cores can access the same shared memory:

1. Core 0 writes value 0xDEADBEEF to address 0x1000
2. Core 1 reads from address 0x1000
3. Asserts that Core 1 sees the value written by Core 0

Reference: core/src/tests/multicore_test.rs:15

#[test]
fn test_shared_memory_access() {
    let manager = CpuManager::new();
    let core0 = manager.get_core(0).expect("Core 0 missing");
    let core1 = manager.get_core(1).expect("Core 1 missing");
    
    let test_addr = 0x1000;
    let test_val = 0xDEADBEEF;
    core0.write_u32(test_addr, test_val);
    let read_val = core1.read_u32(test_addr);
    
    assert_eq!(read_val, test_val);
}

Platform Considerations

macOS Testing

On macOS, especially with Apple Silicon (M1/M2), JIT compilation can have cold-start overhead:

The first test run after build system changes may experience timeouts on macOS. This is normal and subsequent runs should be faster.

The test framework accounts for this with:

• Extended timeout (500ms vs 100ms on Linux)
• Warmup phase before running tests
• Clear messaging when macOS-specific issues occur

Reference: core/src/tests/run.rs:10

Running Tests from GUI

The GUI provides a “Run CPU Tests” button that executes the instruction test suite in a background thread.

1. Launch the GUI

   cargo run
   
2. Click 'Run CPU Tests'

   The button is located in the main window after the splash screen.

3. View results

   Test results appear in the scrollable results panel with:

   • ✅ Green text for passing tests
   • ❌ Red text for failing tests
   • Execution duration for each test

Reference: gui/src/gui/gui.rs:157

Adding New Tests

When adding new functionality, please include tests:

1. For CPU instructions

   Add test cases to core/src/tests/run.rs using the run_test() helper:

   run_test(
       "Your instruction name",
       &[encoded_instruction],
       |cpu| { /* setup */ },
       |cpu| { /* verification */ },
   )
   
2. For other functionality

   Create a new test module in the appropriate location:

   #[cfg(test)]
   mod tests {
       #[test]
       fn test_your_feature() {
           // Test implementation
       }
   }
   

Test Output

Test results include:

• Test name
• Pass/fail status
• Execution duration
• Total summary with pass/fail counts

Example output:

Y NOP - PASS (1.2ms)
Y ADD X1, X1, #2 - PASS (0.8ms)
N SUB X2, X2, #1 - FAIL: Verification failed (PC = 0x1008) (1.5ms)
Total: 9 (8 / 1) time 15.3ms

For debugging failed tests, see the Debugging page for information on enabling trace logging.