In our ongoing research into zero-knowledge virtual machines (zkVMs), we conducted a series of rigorous tests to identify the most suitable solution for the Nescience architecture. This article presents our testing methodology, results, and conclusions while maintaining the original technical depth and analytical perspective.
Introduction
Following our initial exploration of zkVM technologies, we designed a systematic evaluation process to assess multiple candidates against our specific requirements. The testing focused on performance metrics, memory management, and practical usability factors that are critical for real-world implementation.
Why These Specific Candidates?
Our selection process prioritized several key factors:
- True Zero-Knowledge Capability: Each zkVM must demonstrate or approach genuine zero-knowledge proof generation and verification capabilities
- Performance Benchmarks: We sought zkVMs with established performance records, particularly in speed and efficiency metrics
- Specific Features: For Nescience's requirements, features like lookup tables, precompilation support, and recursive proof capabilities were essential
The selected candidates represent the most promising solutions currently available in the zkVM ecosystem.
Tested zkVM Platforms
We evaluated six prominent zkVM solutions:
- SP1: A high-performance open-source zkVM capable of verifying arbitrary Rust programs using Plonky3 with support for recursive proofs
- RISC0: Based on RISC-V architecture, utilizing STARKs with 98-bit security level and supporting multiple programming languages
- Nexus: A modular, scalable open-source zkVM using Nova folding scheme, particularly suited for recursive proofs
- ZkMIPS: A general-purpose verifiable computation infrastructure based on Plonky2 and MIPS microarchitecture
- ZkWASM: Supports unmodified standard WASM bytecode specification, providing flexibility for various language support
- Valida: A STARK-based virtual machine designed for code reuse and maximum prover performance
Testing Methodology
We implemented a two-phase testing process to comprehensively evaluate each zkVM:
Phase 1: Arithmetic Operations
This initial phase assessed each zkVM's ability to handle fundamental arithmetic operations: addition, subtraction, multiplication, division, modulus operations, and square root calculations. We designed tests around heptagonal number calculations, requiring zkVMs to process multiple arithmetic operations simultaneously. This approach allowed us to measure efficiency and speed when handling complex mathematical computations.
Phase 2: Memory Consumption Assessment
The second phase evaluated each zkVM's memory management capabilities under heavy load conditions. We tested various data structures including lists, hash maps, deques, queues, BTreeMaps, hash sets, and binary heaps. Each zkVM underwent testing for:
- Insertion: How quickly the zkVM could add data to structures
- Deletion: Efficiency in handling memory release operations
- Appending: Capability to extend data structures effectively
- Search: Speed and efficiency in retrieving stored data
This phase aimed to identify potential memory bottlenecks and determine each zkVM's ability to manage intensive tasks efficiently.
Hardware Configuration
All testing was conducted on identical hardware specifications:
- CPU: AMD EPYC 7713 "Milan" 64-core processor (128 threads total)
- RAM: 600GiB DDR4 3200MHz ECC RAM distributed across 16 DIMMs
- Host OS: Proxmox 8.3
- Hypervisor: KVM
- Network Layer: Open vSwitch
- Machine Model: Supermicro AS-2024US-TRT
Detailed Results Analysis
SP1 Performance Assessment
SP1 delivered respectable performance despite lacking native zero-knowledge capabilities in its proofs. Memory leakage remained minimal, staying below 700 KB. Interestingly, SP1 consumed more RAM during basic arithmetic tests than during memory allocation tests, demonstrating effective memory handling under load.
Phase 1 - Hept 100 Test:
- Proof Size: 3.108 MB
- Proof Time: 16.95 seconds
Phase 2 - Vec 10000 Test:
- Proof Size: 3.17 MB
- Proof Time: 20.85 seconds
RISC0 Performance Assessment
RISC0 excelled in both proof size and generation time, ranking among the top performers (except when compared to Valida and zkWASM in basic tests). Its memory management proved excellent with minimal leakage below 0.5 MB and controlled RAM consumption under 2.2 GB.
Phase 1 - Hept 100 Test:
- Proof Size: 217.4 KB
- Proof Time: 9.73 seconds
Phase 2 - Vec 10000 Test:
- Proof Size: 217.4 KB
- Proof Time: 16.63 seconds
Based on these results, RISC0 emerged as a strong candidate for Nescience implementation.
Nexus Performance Assessment
Nexus provided interesting insights into zkVMs based on folding schemes. Surprisingly, proof sizes remained consistent across different workloads with no significant memory leakage (below 700 KB). However, despite slightly increased RAM consumption under higher workloads (peaking at 1.2 GB), Nexus performed poorly in memory allocation tests.
Phase 1 - Hept 100 Test:
- Proof Size: 46 MB
- Proof Time: 12.06 seconds
Phase 2 - Vec 10000 Test:
- Proof Size: 46 MB
- Proof Time: 56 minutes
ZkMIPS Performance Assessment
ZkMIPS presented a mixed performance profile. While it demonstrated good proof sizes and generation times in basic tests, this came at the cost of significant RAM usage and memory leakage. Memory allocation tests revealed concerning 6.7 GB memory leakage, with 0.5 GB leakage in basic tests.
Phase 1 - Hept 100 Test:
- Proof Size: 4.3 MB
- Proof Time: 9.32 seconds
Phase 2 - Vec 10000 Test:
- Proof Size: 4.898 MB
- Proof Time: 42.57 seconds
ZkWASM Performance Assessment
ZkWASM delivered the poorest performance in both proof size and generation time across both phases. RAM consumption proved particularly excessive, exceeding 7 GB in basic tests and reaching 57 GB in memory allocation tests.
Phase 1 - Hept 100 Test:
- Proof Size: 18 KB
- Proof Time: 42.7 seconds
Phase 2 - Vec 10000 Test:
- Proof Size: 334 KB
- Proof Time: 323 seconds
Valida Performance Assessment
Valida demonstrated exceptional performance in proof generation speed and size, producing 280 KB proofs in less than 1 second. However, limited Rust support prevented comprehensive performance analysis. Valida currently uses an LLVM backend to compile Rust, converting LLVM IR to its C/C++ implementation, which causes errors with Rust-specific data structures or dependencies.
👉 Explore advanced zkVM implementation strategies
Comparative Results Summary
Phase 1 Performance Metrics
| zkVM | Proof Time | Proof Size | Peak RAM Consumption | Memory Leakage |
|---|---|---|---|---|
| SP1 | 16.95 s | 3.108 MB | 2.1 GB | 656.8 KB |
| RISC0 | 9.73 s | 217.4 KB | 1.9 GB | 470.5 KB |
| Nexus | 12.06 s | 46 MB | 9.7 MB | 646.5 KB |
| ZkMIPS | 9.32 s | 4.3 MB | 17.3 GB | 453.8 MB |
| ZkWASM | 42.7 s | 18 KB | 8.2 GB | 259.4 KB |
| Valida | < 1 s | 280 KB | N/A | N/A |
Phase 2 Performance Metrics
| zkVM | Proof Time | Proof Size | Peak RAM Consumption | Memory Leakage |
|---|---|---|---|---|
| SP1 | 20.85 s | 3.17 MB | 1.9 GB | 616 KB |
| RISC0 | 16.63 s | 217.4 KB | 2.3 GB | 485.3 KB |
| Nexus | 56 m | 46 MB | 1.9 GB | 616 KB |
| ZkMIPS | 42.57 s | 4.898 MB | 18.9 GB | 6.9 GB |
| ZkWASM | 323 s | 334 KB | 58.8 GB | 259.4 KB |
| Valida | N/A | N/A | N/A | N/A |
Conclusion and Recommendations
After extensive evaluation of six zkVM candidates, RISC0 emerged as the optimal choice for our requirements. It demonstrated excellent proof generation times and sizes while maintaining reasonable memory footprint. With robust zero-knowledge proof capabilities and support for multiple programming languages, it effectively meets our project needs for privacy, performance, and flexibility.
Valida shows significant potential for high proof performance but remains in early development stages with Rust integration issues. Current LLVM IR conversion limitations prevent handling complex memory interactions, making it unsuitable for immediate implementation. However, as development matures, Valida may become a strong alternative.
SP1, despite initial promise, failed to meet zero-knowledge proof requirements. Its satisfactory performance in arithmetic operations was overshadowed by lacking ZK capabilities, which are crucial for our privacy-first objectives.
Nexus showed consistent proof sizes and manageable memory usage but demonstrated poor performance in memory-intensive tasks. zkMIPS offered reasonable proof times but suffered from critical memory issues that cannot be overlooked. ZkWASM delivered the weakest overall results with excessive RAM consumption making it impractical for production use.
👉 Learn more about optimizing zero-knowledge proof systems
Frequently Asked Questions
What distinguishes zkVMs from traditional virtual machines?
zkVMs incorporate zero-knowledge proof capabilities that allow verification of computation correctness without revealing underlying data. This privacy-preserving feature enables trustless verification of execution results while maintaining data confidentiality, unlike traditional VMs that focus solely on execution without built-in privacy features.
How important is memory management in zkVM selection?
Memory management is critically important because inefficient memory handling can lead to performance bottlenecks and scalability issues. Our tests revealed significant variations in memory leakage and consumption patterns across different zkVMs, directly impacting their suitability for production environments with intensive computational requirements.
Why was RISC0 selected over other candidates?
RISC0 demonstrated the best balance of performance metrics including proof generation speed, proof size efficiency, and memory management. Its support for multiple programming languages and robust zero-knowledge capabilities made it the most suitable choice for our specific architecture requirements and future scalability needs.
Can these test results be replicated on different hardware?
While absolute performance numbers may vary across different hardware configurations, the relative performance differences between zkVM solutions should remain consistent. The testing methodology provides a framework for comparative evaluation regardless of specific hardware implementation.
What future developments might change these recommendations?
zkVM technology is rapidly evolving with ongoing improvements in proof systems, hardware acceleration support, and language compatibility. Valida's development progress particularly warrants monitoring, as it may become a strong contender once Rust integration issues are resolved and the platform matures further.
How does proof size impact practical implementation?
Smaller proof sizes reduce storage requirements and transmission costs, which is crucial for blockchain applications where on-chain storage is expensive. Proof generation time directly affects user experience and system throughput, making both metrics critical for practical deployment considerations.
Future Research Directions
We plan to compare RISC0 and SP1 with CUDA acceleration capabilities in future testing rounds. Ideally, more zkVM platforms will incorporate similar acceleration capabilities, enabling more comprehensive and equitable cross-platform comparisons. The continuous evolution of zkVM technology promises exciting developments in performance optimization and feature expansion.
The field of zero-knowledge virtual machines continues to advance rapidly, with new solutions and improvements emerging regularly. Our testing provides a snapshot of current capabilities while establishing a framework for ongoing evaluation as the technology matures and new contenders enter the space.