gists/WARP.md

WARP.md

This file provides guidance to WARP (warp.dev) when working with code in this repository.

Repository Overview

This is a collection of utility scripts, tools, and gists organized for cybersecurity, forensics, data analysis, and system administration tasks. The repository contains standalone utilities rather than a cohesive application, with scripts written in Python, Bash, Go, JavaScript, PowerShell, and C.

Key Directory Structure

• codegrab/ - Main collection of security and analysis tools
  • ctf/ - CTF challenge solving scripts
  • puzzlebox/ - 3D puzzle solving algorithms with visualization
• tools/ - System utilities and data processing tools
• config/ - System configuration and installation scripts
• systemscripts/ - System administration and environment setup
  • proxy/ - Network proxy configuration utilities
• dockerfiles/ - Docker container build scripts
• collected/ - Archive of older utilities with documentation

Common Development Tasks

Running Security Analysis Tools

Most security tools are standalone and follow this pattern:

# VirusTotal scanning
./codegrab/scan_vt.py <filename>

# Import hash calculation
python3 codegrab/imphash.py <pe_file>

# Network analysis
./codegrab/scapy_arp.py
./codegrab/simple_portscan.py

Data Processing Utilities

# Hash utilities for archives
python3 tools/libarchivesum.py archive.zip

# Unicode character analysis
echo "text" | python3 tools/unum.py

# Domain extraction from URLs
cat urls.txt | python3 tools/domgrep.py

# File organization by MIME type
python3 tools/rename.mime.py

Docker Environment Management

# Backup Docker Compose stacks
./tools/backup_docker.sh docker-compose.yml

# Restore Docker environments
./tools/restore_docker.sh

# Build forensics containers
./dockerfiles/build_kali.sh

System Configuration

# Install dependencies and configure environment
./config/install.sh

# Proxy configuration
./systemscripts/proxy/get_proxy.sh
./systemscripts/proxy/update_apt_proxy.sh

Architecture and Patterns

Security Tools Pattern

Most security utilities in codegrab/ follow this pattern:

• Standalone executables with shebang
• Take file paths or stdin as input
• Output results in structured format (often CSV-like with custom separators)
• Use external APIs (VirusTotal, etc.) with API keys from ~/.virustotal_api_key

Data Processing Pattern

Tools in tools/ directory typically:

• Accept multiple file inputs via command line arguments
• Use argparse for option handling
• Support multiple hash algorithms or processing modes
• Include error handling for malformed inputs

System Scripts Pattern

Scripts in systemscripts/ are designed for:

• Environment detection and configuration
• Proxy and network setup automation
• Service management and monitoring
• Display and hardware management

Specialized Solvers

The puzzlebox/ directory contains algorithmic solvers featuring:

• 3D spatial problem solving with numpy
• Visualization using matplotlib
• Recursive backtracking algorithms
• Multi-processing optimization variants

Key Dependencies

The repository relies on various Python packages that should be available:

• Security: pefile, requests, scapy
• Data Processing: libarchive-c, openpyxl, visidata
• Scientific: numpy, matplotlib, scipy
• Forensics: AnalyzeMFT, pymisp
• System: ntplib, mac-vendor-lookup, dateparser

API Keys and Configuration

Several tools expect API keys in home directory files:

• ~/.virustotal_api_key - VirusTotal API access
• Tools may also use environment variables for proxy configuration (http_proxy, etc.)

Testing and Validation

Tools are typically tested individually:

# Test with sample data
python3 codegrab/chechsqlite.py sample.db
python3 tools/quickchardet.py sample.txt

# Validate with CTF challenges
python3 codegrab/ctf/solve.py

Development Notes

• Most utilities are designed as single-file executables for easy deployment
• Scripts include minimal error handling suitable for command-line usage
• Many tools output to stdout in formats suitable for piping to other commands
• Docker-based tools assume availability of container runtime
• Forensics tools may require elevated privileges for certain operations