Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Development Setup

This guide covers setting up a development environment for contributing to libmagic-rs, including tools, workflows, and best practices.

Current Implementation Status

Project Phase: Active Development with Solid Foundation

Completed Components ✅

  • Core AST Structures: Complete with 29 comprehensive unit tests
  • Parser Components: Numbers, offsets, operators, values (50 unit tests)
  • CLI Framework: Basic command-line interface with clap
  • Code Quality: Zero-warnings policy with comprehensive linting
  • Serialization: Full serde support for all data structures
  • Memory Safety: Zero unsafe code with bounds checking

In Progress 🔄

  • Complete Magic File Parser: Integration of parsing components
  • Rule Evaluation Engine: Offset resolution and type interpretation
  • Memory-Mapped I/O: Efficient file access with memmap2
  • Output Formatters: Text and JSON result formatting

Test Coverage

Current test suite includes 79 passing unit tests:

# Run current test suite
cargo test
# Output: running 79 tests ... test result: ok. 79 passed; 0 failed

Test Categories:

  • AST structure tests (29 tests)
  • Parser component tests (50 tests)
  • Serialization round-trip tests
  • Edge case and boundary value tests
  • Error condition handling tests

Prerequisites

Required Tools

  • Rust 1.85+ with the 2021 edition
  • Git for version control
  • Cargo (included with Rust)
# Enhanced test runner
cargo install cargo-nextest

# Auto-rebuild on file changes
cargo install cargo-watch

# Code coverage
cargo install cargo-llvm-cov

# Security auditing
cargo install cargo-audit

# Dependency analysis
cargo install cargo-tree

# Documentation tools
cargo install mdbook  # For this documentation

Environment Setup

1. Clone the Repository

git clone https://github.com/EvilBit-Labs/libmagic-rs.git
cd libmagic-rs

2. Verify Setup

# Check Rust version
rustc --version  # Should be 1.85+

# Verify project builds
cargo check

# Run tests
cargo test

# Check linting passes
cargo clippy -- -D warnings

3. IDE Configuration

VS Code

Recommended extensions:

  • rust-analyzer: Rust language server
  • CodeLLDB: Debugging support
  • Better TOML: TOML syntax highlighting
  • Error Lens: Inline error display

Settings (.vscode/settings.json):

{
  "rust-analyzer.check.command": "clippy",
  "rust-analyzer.check.extraArgs": [
    "--",
    "-D",
    "warnings"
  ],
  "rust-analyzer.cargo.features": "all"
}

Other IDEs

  • IntelliJ IDEA: Use the Rust plugin
  • Vim/Neovim: Configure with rust-analyzer LSP
  • Emacs: Use rustic-mode with lsp-mode

Development Workflow

Daily Development

# Start development session
cargo watch -x check -x test

# In another terminal, make changes and see results automatically

Code Quality Checks

# Format code (required before commits)
cargo fmt

# Check for issues (must pass)
cargo clippy -- -D warnings

# Run all tests
cargo nextest run  # or cargo test

# Check documentation
cargo doc --document-private-items

Testing Strategy

# Run specific test modules
cargo test ast_structures
cargo test parser
cargo test evaluator

# Run tests with output
cargo test -- --nocapture

# Run ignored tests (if any)
cargo test -- --ignored

# Test documentation examples
cargo test --doc

Project Standards

Code Style

The project enforces strict code quality standards:

Linting Configuration

See Cargo.toml for the complete linting setup. Key rules:

  • No unsafe code: unsafe_code = "forbid"
  • Zero warnings: warnings = "deny"
  • Comprehensive clippy: Pedantic, nursery, and security lints enabled
  • No unwrap/panic: unwrap_used = "deny", panic = "deny"

Formatting

# Format all code (required)
cargo fmt

# Check formatting without changing files
cargo fmt -- --check

Documentation Standards

Code Documentation

All public APIs must have rustdoc comments:

#![allow(unused)]
fn main() {
/// Parses a magic file into an AST
///
/// This function reads a magic file from the given path and parses it into
/// a vector of `MagicRule` structures that can be used for file type detection.
///
/// # Arguments
///
/// * `path` - Path to the magic file to parse
///
/// # Returns
///
/// Returns `Ok(Vec<MagicRule>)` on success, or `Err(LibmagicError)` if parsing fails.
///
/// # Errors
///
/// This function will return an error if:
/// - The file cannot be read
/// - The magic file syntax is invalid
/// - Memory allocation fails
///
/// # Examples
///
/// ```rust,no_run
/// use libmagic_rs::parser::parse_magic_file;
///
/// let rules = parse_magic_file("magic.db")?;
/// println!("Loaded {} rules", rules.len());
/// # Ok::<(), Box<dyn std::error::Error>>(())
/// ```
pub fn parse_magic_file<P: AsRef<Path>>(path: P) -> Result<Vec<MagicRule>> {
    // Implementation
}
}

Module Documentation

Each module should have comprehensive documentation:

#![allow(unused)]
fn main() {
//! Magic file parser module
//!
//! This module handles parsing of magic files into an Abstract Syntax Tree (AST)
//! that can be evaluated against file buffers for type identification.
//!
//! # Magic File Format
//!
//! Magic files use a simple DSL to describe file type detection rules:
//!
//! ```text
//! # ELF files
//! 0    string    \x7fELF    ELF
//! >4   byte      1          32-bit
//! >4   byte      2          64-bit
//! ```
//!
//! # Examples
//!
//! ```rust,no_run
//! use libmagic_rs::parser::parse_magic_file;
//!
//! let rules = parse_magic_file("magic.db")?;
//! # Ok::<(), Box<dyn std::error::Error>>(())
//! ```
}

Testing Standards

Unit Tests

Every module should have comprehensive unit tests:

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_basic_functionality() {
        // Test basic case
        let result = function_under_test();
        assert_eq!(result, expected_value);
    }

    #[test]
    fn test_error_conditions() {
        // Test error handling
        let result = function_that_should_fail();
        assert!(result.is_err());
    }

    #[test]
    fn test_edge_cases() {
        // Test boundary conditions
        // Empty inputs, maximum values, etc.
    }
}
}

Integration Tests

Place integration tests in the tests/ directory:

#![allow(unused)]
fn main() {
// tests/integration_test.rs
use libmagic_rs::*;

#[test]
fn test_end_to_end_workflow() {
    // Test complete workflows
    let db = MagicDatabase::load_from_file("third_party/magic.mgc").unwrap();
    let result = db
        .evaluate_file("third_party/tests/elf64.testfile")
        .unwrap();
    assert_eq!(result.description, "ELF 64-bit LSB executable");
}
}

Error Handling

Use the project’s error types consistently:

#![allow(unused)]
fn main() {
use thiserror::Error;

#[derive(Debug, Error)]
pub enum ModuleError {
    #[error("Invalid input: {0}")]
    InvalidInput(String),

    #[error("Processing failed: {reason}")]
    ProcessingFailed { reason: String },

    #[error("IO error: {0}")]
    IoError(#[from] std::io::Error),
}

pub type Result<T> = std::result::Result<T, ModuleError>;
}

Contribution Workflow

1. Issue Creation

Before starting work:

  • Check existing issues and discussions
  • Create an issue describing the problem or feature
  • Wait for maintainer feedback on approach

2. Branch Creation

# Create feature branch
git checkout -b feature/descriptive-name

# Or for bug fixes
git checkout -b fix/issue-description

3. Development Process

# Make changes following the standards above
# Run checks frequently
cargo watch -x check -x test

# Before committing
cargo fmt
cargo clippy -- -D warnings
cargo test

4. Commit Guidelines

Use conventional commit format:

# Feature commits
git commit -m "feat(parser): add support for indirect offsets"

# Bug fixes
git commit -m "fix(evaluator): handle buffer overflow in string reading"

# Documentation
git commit -m "docs(api): add examples for MagicRule creation"

# Tests
git commit -m "test(ast): add comprehensive serialization tests"

5. Pull Request Process

  1. Push branch: git push origin feature/descriptive-name
  2. Create PR with:
    • Clear description of changes
    • Reference to related issues
    • Test coverage information
    • Breaking change notes (if any)
  3. Address feedback from code review
  4. Ensure CI passes all checks

Debugging

Logging

Use the log crate for debugging:

#![allow(unused)]
fn main() {
use log::{debug, error, info, warn};

pub fn parse_rule(input: &str) -> Result<MagicRule> {
    debug!("Parsing rule: {}", input);

    let result = do_parsing(input)?;

    info!("Successfully parsed rule: {}", result.message);
    Ok(result)
}
}

Run with logging:

RUST_LOG=debug cargo test
RUST_LOG=libmagic_rs=trace cargo run

Debugging Tests

# Run single test with output
cargo test test_name -- --nocapture

# Debug with lldb/gdb
cargo test --no-run
lldb target/debug/deps/libmagic_rs-<hash>

Performance Profiling

# Install profiling tools
cargo install cargo-flamegraph

# Profile specific benchmarks
cargo flamegraph --bench evaluation_bench

# Memory profiling with valgrind
cargo build
valgrind --tool=massif target/debug/rmagic large_file.bin

Continuous Integration

The project uses GitHub Actions for CI. Local checks should match CI:

# Run the same checks as CI
cargo fmt -- --check
cargo clippy -- -D warnings
cargo test
cargo doc --document-private-items

Release Process

For maintainers:

Version Bumping

# Update version in Cargo.toml
# Update CHANGELOG.md
# Commit changes
git commit -m "chore: bump version to 0.2.0"
git tag v0.2.0
git push origin main --tags

Documentation Updates

# Update documentation
mdbook build docs/
# Deploy to GitHub Pages (automated)

This development setup ensures high code quality, comprehensive testing, and smooth collaboration across the project.