irt/docker_file_analysis
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Major repository cleanup and enhancement

Browse Source 
- Reorganize documentation: moved old docs to docs/ directory
- Add comprehensive README.md with build options and usage guide
- Add detailed CONTRIBUTING.md with help content management guide
- Create Makefile for automated building and testing
- Add Dockerfile.scratch for building from Ubuntu 20.04 base
- Enhance all Dockerfiles with PowerShell + PSScriptAnalyzer
- Add modern shells: zsh (with plugins) and fish (with config)
- Add modern CLI tools: fd-find, ripgrep, fzf
- Create comprehensive help system with cheat/TLDR/fish completions
- Add helper scripts for help content management and coverage checking
- Fix Dockerfile.remnux script references
- Support three build variants: upstream (REMnux), scratch (Ubuntu), kali

Build options:
  - make build-upstream: Fast, uses REMnux upstream (recommended)
  - make build-scratch: Full control, builds from Ubuntu 20.04
  - make build-kali: Legacy Kali Linux base

Features:
  - PowerShell with PSScriptAnalyzer module
  - Modern shells (zsh, fish) with custom configurations
  - Enhanced help system (cheat sheets, TLDR pages, fish completions)
  - Help coverage checking and bulk import tools
  - Comprehensive documentation for users and contributors
This commit is contained in:
Tobias Kessels
2025-10-01 11:45:56 +02:00
parent 6bfcfd7935
commit b98aaee3e0
27 changed files with 5000 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+292 -37
View File
@@ -1,49 +1,304 @@
# docker_file_analysis
# File Analysis Container
## REMnux-Based File Analysis Container
A comprehensive Docker-based toolkit for malware and file forensics analysis, featuring an extensive offline help system and modern shell environments.
This container is now based on the REMnux malware analysis toolkit, providing a comprehensive set of tools for file analysis, especially PDFs and malware samples.
## 🎯 Quick Start
## Usage
```bash
# REMnux-based version
docker build -f Dockerfile.remnux -t tabledevil/file-analysis:remnux .
# Clone the repository
git clone https://github.com/tabledevil/docker_file_analysis.git
cd docker_file_analysis
# Build using REMnux upstream (recommended)
make build-upstream
# Run the container
docker run -it --rm -v "$(pwd):/data" tabledevil/file-analysis:remnux
# Original Kali-based version (legacy)
docker run -it --rm -v "$(pwd):/data" tabledevil/file-analysis
# Inside the container, get help
fhelp
```
## Included Tools (REMnux Base + Additional)
## 📦 Build Options
### PDF Analysis Suite (from REMnux)
* **peepdf** - PDF analysis framework with JavaScript detection
* **pdf-parser.py** - Extract and analyze PDF elements (Didier Stevens)
* **pdfid.py** - Quick PDF structure overview (Didier Stevens)
* **origami** - Ruby gem suite (pdfcop, pdfextract, pdfmetadata)
* **pdftk-java** - PDF manipulation and flattening
* **qpdf** - PDF manipulation (merge, convert, transform)
* **pdfresurrect** - Extract previous versions from PDFs
* **pdftool** - Analyze PDF incremental updates
This project offers **three different build strategies** to suit your needs:
### Malware Analysis (Additional)
* **capa** - Malware capability detection (Mandiant)
* **box-js** - JavaScript sandbox analysis
* **oletools** - Office document analysis suite
* oledump.py
* rtfdump.py
* emldump.py
* and more
### 1. **REMnux Upstream** (Recommended)
Uses the official `remnux/remnux-distro` image as a base and adds enhancements.
### Data Analysis & Utilities (Additional)
* **visidata** - Data exploration and analysis
* **unfurl** - URL and data analysis (DFIR)
* **base64dump** - Base64 decoder (Didier Stevens)
* **tesseract** - OCR text extraction
* **exiftool** - Metadata extraction
```bash
make build-upstream
# OR
docker build -f Dockerfile.remnux -t tabledevil/file-analysis:remnux .
```
### System Tools
* **mc** - Midnight Commander file manager
* **p7zip-full** - Archive utilities
* All standard REMnux tools and utilities
**Advantages:**
- Fast build (uses pre-built REMnux image)
- Includes all REMnux tools and configurations
- Regular updates from upstream
- Production-ready
### 2. **From Scratch** (Full Control)
Builds a REMnux-like environment from Ubuntu 20.04 base, mimicking the official build.
```bash
make build-scratch
# OR
docker build -f Dockerfile.scratch -t tabledevil/file-analysis:scratch .
```
**Advantages:**
- Full control over every package and configuration
- Understand exactly what's installed
- Customize the base system
- Smaller final image (optional)
### 3. **Kali Base** (Legacy)
Original implementation using Kali Linux rolling as the base.
```bash
make build-kali
# OR
docker build -f Dockerfile -t tabledevil/file-analysis:kali .
```
**Advantages:**
- Access to Kali Linux security tools
- Different package ecosystem
- Alternative to REMnux
## 🚀 Usage
### Basic Analysis Workflow
```bash
# Start the container with a directory containing files to analyze
docker run -it --rm -v "/path/to/suspicious/files:/data" tabledevil/file-analysis:remnux
# Inside the container:
# Get help on available tools
fhelp
# Find PDF analysis tools
fhelp tools pdf
# Quick command examples for a specific tool
fhelp cheat pdfid.py
# Analyze a PDF
pdfid.py suspicious.pdf
pdf-parser.py suspicious.pdf
# Interactive cheat sheet browser
fhelp examples
# Switch to zsh or fish for better interactivity
zsh
# or
fish
```
### Alternative Shells
The container includes three shells with different features:
- **bash** (default) - Traditional, reliable
- **zsh** - Advanced completion, history search, plugins
- **fish** - Friendly syntax, autosuggestions
```bash
# Try zsh
zsh
# Try fish
fish
```
## 📚 Comprehensive Help System
The container features an **offline-first help system** with multiple layers:
### 1. Command-Line Help (`fhelp`)
```bash
fhelp # Main help menu
fhelp tools pdf # Find PDF analysis tools
fhelp cheat <tool> # Quick examples for a tool
fhelp tldr <tool> # Simplified man pages
fhelp examples # Browse all cheat sheets interactively
fhelp pdf # PDF analysis workflow guide
```
### 2. Tool Coverage
The help system includes documentation for **100+ analysis tools**:
- **PDF Analysis:** pdfid, pdf-parser, peepdf, pdftk, qpdf, pdfresurrect, origami suite
- **Office Documents:** oledump, rtfdump, oletools, emldump
- **Malware Analysis:** capa, box-js, strings, vivisect
- **File Inspection:** exiftool, file, binwalk, hexdump
- **Scripting:** python, ruby, perl, powershell (with PSScriptAnalyzer)
- **Data Analysis:** visidata, jq, yq, sqlite3
- **System Tools:** fd-find, ripgrep, zsh, fish
### 3. Help Content Types
- **Cheat Sheets** - Quick command examples and common patterns
- **TLDR Pages** - Simplified, example-focused documentation
- **Fish Completions** - Smart command-line autocompletion
- **Workflow Guides** - Multi-tool analysis procedures
## 🛠️ Adding & Modifying Help Content
See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed instructions on:
- Adding cheat sheets for new tools
- Creating TLDR pages
- Writing fish shell completions
- Importing bulk cheatsheets from markdown
- Checking help coverage for installed tools
### Quick Example: Add a Cheat Sheet
```bash
# Inside the container:
cat > /opt/cheatsheets/personal/mytool << 'EOF'
# mytool - Description
# Basic usage
mytool file.txt
# Advanced options
mytool -v --output result.txt input.txt
EOF
# Test it
fhelp cheat mytool
```
## 🔧 Included Tools
### PDF Analysis Suite
- **peepdf** - Interactive PDF analysis with JavaScript detection
- **pdf-parser.py** - Extract and analyze PDF elements (Didier Stevens)
- **pdfid.py** - Quick PDF structure overview
- **pdftk** - PDF manipulation and transformation
- **qpdf** - PDF inspection and transformation
- **origami** - Ruby suite (pdfcop, pdfextract, pdfmetadata)
- **pdfresurrect** - Extract previous versions from PDFs
### Malware Analysis
- **capa** - Detect malware capabilities (Mandiant)
- **box-js** - JavaScript sandbox
- **oletools** - Office document analysis (oledump, rtfdump, emldump)
- **vivisect** - Malware analysis framework
- **strings** - Extract printable strings
- **upx** - Executable packer/unpacker
### Modern Shells & Tools
- **PowerShell** - Cross-platform PowerShell with PSScriptAnalyzer
- **zsh** - With autosuggestions and syntax highlighting
- **fish** - Friendly interactive shell
- **fd-find** - Modern, fast file finder
- **ripgrep** - Ultra-fast recursive grep
### Data Analysis
- **visidata** - Terminal spreadsheet and data explorer
- **unfurl** - URL and forensics data analyzer
- **jq** - JSON processor
- **sqlite3** - Database analysis
### File Inspection
- **exiftool** - Metadata extraction
- **binwalk** - Firmware analysis
- **hexdump** / **xxd** - Binary viewers
- **file** - File type identification
## 📁 Repository Structure
```
docker_file_analysis/
├── Dockerfile # Kali-based build (legacy)
├── Dockerfile.remnux # REMnux upstream build (recommended)
├── Dockerfile.scratch # Build from Ubuntu base (full control)
├── Makefile # Build automation
├── README.md # This file
├── CONTRIBUTING.md # How to add/modify help content
├── WARP.md # WARP AI assistant context
├── files/ # Container configuration files
│ ├── README # Welcome message shown on login
│ ├── command_help # Detailed command examples
│ ├── zshrc # Zsh shell configuration
│ └── fish_config.fish # Fish shell configuration
├── scripts/ # Helper scripts
│ ├── fhelp # Main help system
│ ├── create-offline-help-system.sh # Build help database
│ ├── add-tool-cheats.sh # Add default cheat sheets
│ ├── import-remnux-cheatsheets.sh # Import bulk cheatsheets
│ ├── convert-remnux-cheats.py # Convert markdown to cheat format
│ ├── check-help-coverage.sh # Verify help coverage
│ └── find-tool # Search for tools
├── cheatsheets/ # Custom cheat sheets
│ ├── pdf-analysis.cheat
│ ├── malware-analysis.cheat
│ └── system-utilities.cheat
├── docs/ # Additional documentation
└── tests/ # Test scripts
```
## 🧪 Testing
```bash
# Test all builds
make test
# Test specific build
docker run --rm tabledevil/file-analysis:remnux fhelp cheat pdfid
# Run help coverage check
docker run --rm tabledevil/file-analysis:remnux check-help-coverage.sh
```
## 🐳 Docker Hub
Pre-built images are available:
```bash
# Pull the latest REMnux-based image
docker pull tabledevil/file-analysis:remnux
# Pull the Kali-based image (legacy)
docker pull tabledevil/file-analysis:latest
```
## 🤝 Contributing
Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for:
- Adding new tools
- Improving help content
- Adding cheat sheets and TLDR pages
- Enhancing shell configurations
- Reporting bugs
## 📝 License
This project packages various open-source tools. Please respect individual tool licenses.
## 🙏 Acknowledgments
- [REMnux](https://remnux.org/) - Malware analysis toolkit
- [Didier Stevens](https://blog.didierstevens.com/) - PDF analysis tools
- [Mandiant](https://www.mandiant.com/) - CAPA malware analysis
- [cheat](https://github.com/cheat/cheat) - Cheat sheet system
- [tldr](https://tldr.sh/) - Simplified man pages
## 📮 Support
- **Issues**: [GitHub Issues](https://github.com/tabledevil/docker_file_analysis/issues)
- **Discussions**: [GitHub Discussions](https://github.com/tabledevil/docker_file_analysis/discussions)
---
**Security Note**: This container is designed for analyzing potentially malicious files. Always run it with appropriate isolation and never execute untrusted code outside the container.