# WARP.md

This file provides guidance to WARP (warp.dev) when working with code in this repository.

## Repository Overview

This is a collection of utility scripts, tools, and gists organized for cybersecurity, forensics, data analysis, and system administration tasks. The repository contains standalone utilities rather than a cohesive application, with scripts written in Python, Bash, Go, JavaScript, PowerShell, and C.

## Key Directory Structure

- **`codegrab/`** - Main collection of security and analysis tools
  - `ctf/` - CTF challenge solving scripts
  - `puzzlebox/` - 3D puzzle solving algorithms with visualization
- **`tools/`** - System utilities and data processing tools
- **`config/`** - System configuration and installation scripts
- **`systemscripts/`** - System administration and environment setup
  - `proxy/` - Network proxy configuration utilities
- **`dockerfiles/`** - Docker container build scripts
- **`collected/`** - Archive of older utilities with documentation

## Common Development Tasks

### Running Security Analysis Tools

Most security tools are standalone and follow this pattern:
```bash
# VirusTotal scanning
./codegrab/scan_vt.py <filename>

# Import hash calculation
python3 codegrab/imphash.py <pe_file>

# Network analysis
./codegrab/scapy_arp.py
./codegrab/simple_portscan.py
```

### Data Processing Utilities

```bash
# Hash utilities for archives
python3 tools/libarchivesum.py archive.zip

# Unicode character analysis
echo "text" | python3 tools/unum.py

# Domain extraction from URLs
cat urls.txt | python3 tools/domgrep.py

# File organization by MIME type
python3 tools/rename.mime.py
```

### Docker Environment Management

```bash
# Backup Docker Compose stacks
./tools/backup_docker.sh docker-compose.yml

# Restore Docker environments
./tools/restore_docker.sh

# Build forensics containers
./dockerfiles/build_kali.sh
```

### System Configuration

```bash
# Install dependencies and configure environment
./config/install.sh

# Proxy configuration
./systemscripts/proxy/get_proxy.sh
./systemscripts/proxy/update_apt_proxy.sh
```

## Architecture and Patterns

### Security Tools Pattern
Most security utilities in `codegrab/` follow this pattern:
- Standalone executables with shebang
- Take file paths or stdin as input
- Output results in structured format (often CSV-like with custom separators)
- Use external APIs (VirusTotal, etc.) with API keys from `~/.virustotal_api_key`

### Data Processing Pattern
Tools in `tools/` directory typically:
- Accept multiple file inputs via command line arguments
- Use argparse for option handling
- Support multiple hash algorithms or processing modes
- Include error handling for malformed inputs

### System Scripts Pattern
Scripts in `systemscripts/` are designed for:
- Environment detection and configuration
- Proxy and network setup automation
- Service management and monitoring
- Display and hardware management

### Specialized Solvers
The `puzzlebox/` directory contains algorithmic solvers featuring:
- 3D spatial problem solving with numpy
- Visualization using matplotlib
- Recursive backtracking algorithms
- Multi-processing optimization variants

## Key Dependencies

The repository relies on various Python packages that should be available:
- **Security**: `pefile`, `requests`, `scapy`
- **Data Processing**: `libarchive-c`, `openpyxl`, `visidata` 
- **Scientific**: `numpy`, `matplotlib`, `scipy`
- **Forensics**: `AnalyzeMFT`, `pymisp`
- **System**: `ntplib`, `mac-vendor-lookup`, `dateparser`

## API Keys and Configuration

Several tools expect API keys in home directory files:
- `~/.virustotal_api_key` - VirusTotal API access
- Tools may also use environment variables for proxy configuration (`http_proxy`, etc.)

## Testing and Validation

Tools are typically tested individually:
```bash
# Test with sample data
python3 codegrab/chechsqlite.py sample.db
python3 tools/quickchardet.py sample.txt

# Validate with CTF challenges
python3 codegrab/ctf/solve.py
```

## Development Notes

- Most utilities are designed as single-file executables for easy deployment
- Scripts include minimal error handling suitable for command-line usage
- Many tools output to stdout in formats suitable for piping to other commands
- Docker-based tools assume availability of container runtime
- Forensics tools may require elevated privileges for certain operations