Major repository cleanup and enhancement
- Reorganize documentation: moved old docs to docs/ directory - Add comprehensive README.md with build options and usage guide - Add detailed CONTRIBUTING.md with help content management guide - Create Makefile for automated building and testing - Add Dockerfile.scratch for building from Ubuntu 20.04 base - Enhance all Dockerfiles with PowerShell + PSScriptAnalyzer - Add modern shells: zsh (with plugins) and fish (with config) - Add modern CLI tools: fd-find, ripgrep, fzf - Create comprehensive help system with cheat/TLDR/fish completions - Add helper scripts for help content management and coverage checking - Fix Dockerfile.remnux script references - Support three build variants: upstream (REMnux), scratch (Ubuntu), kali Build options: - make build-upstream: Fast, uses REMnux upstream (recommended) - make build-scratch: Full control, builds from Ubuntu 20.04 - make build-kali: Legacy Kali Linux base Features: - PowerShell with PSScriptAnalyzer module - Modern shells (zsh, fish) with custom configurations - Enhanced help system (cheat sheets, TLDR pages, fish completions) - Help coverage checking and bulk import tools - Comprehensive documentation for users and contributors
This commit is contained in:
Tobias Kessels
154
docs/HELP_SYSTEM_ENHANCED.md
Normal file
154
docs/HELP_SYSTEM_ENHANCED.md
Normal file
@@ -0,0 +1,154 @@
|
||||
# Enhanced File Analysis Help System
|
||||
|
||||
## 🎯 Overview
|
||||
|
||||
This document describes the comprehensive improvements made to the container's help system, addressing all the issues you encountered and providing a robust, offline-capable help experience.
|
||||
|
||||
## ✅ Problems Fixed
|
||||
|
||||
### 1. **Navi Configuration Issues**
|
||||
- **Problem**: `invalid preview window layout: up:2:nohidden` errors
|
||||
- **Solution**: Created proper navi configuration file at `/root/.config/navi/config.yaml` with correct fzf settings
|
||||
- **Result**: Navi now works without preview window errors
|
||||
|
||||
### 2. **Inaccurate Tool Information**
|
||||
- **Problem**: Tools listed that don't exist (evince, ioc_parser), unclear command names
|
||||
- **Solution**: Created curated tools database with only verified, available tools and their exact command names
|
||||
- **Result**: `fhelp tools` now shows accurate, actionable information
|
||||
|
||||
### 3. **Missing Cheat Sheets**
|
||||
- **Problem**: Limited or missing practical examples for PDF/malware analysis
|
||||
- **Solution**: Downloaded existing cheat sheets from popular repos and enhanced them with analysis-specific examples
|
||||
- **Result**: Comprehensive cheat sheets for analysis workflows with exact commands
|
||||
|
||||
### 4. **TLDR Cache Issues**
|
||||
- **Problem**: TLDR not finding analysis-specific tools
|
||||
- **Solution**: Created custom TLDR pages for pdfid.py, pdf-parser.py, peepdf, capa
|
||||
- **Result**: `fhelp quick <tool>` now works for specialized analysis tools
|
||||
|
||||
### 5. **Command Clarity**
|
||||
- **Problem**: Users couldn't find the exact command to run (e.g., confusion about ioc_parser)
|
||||
- **Solution**: All help now shows exact command syntax with practical examples
|
||||
- **Result**: Clear, copy-pasteable commands for all analysis scenarios
|
||||
|
||||
## 🛠️ Enhanced Components
|
||||
|
||||
### 1. **Accurate Tools Database** (`/opt/remnux-docs/tools.db`)
|
||||
- Only verified, available tools
|
||||
- Exact command names (not just descriptions)
|
||||
- Usage examples for each tool
|
||||
- Availability status indicators
|
||||
|
||||
### 2. **Enhanced Cheat Sheets** (`/opt/cheatsheets/`)
|
||||
- Downloads existing cheat sheets from `cheat/cheatsheets` repo
|
||||
- Adds analysis-specific examples on top
|
||||
- Covers PDF analysis workflow, malware analysis workflow
|
||||
- Security-focused examples (PDF flattening, password removal)
|
||||
|
||||
### 3. **Interactive Navi Sheets** (`/opt/navi-cheats/`)
|
||||
- Dynamic command completion with file discovery
|
||||
- Interactive parameter selection
|
||||
- Context-aware suggestions
|
||||
|
||||
### 4. **Custom TLDR Pages** (`/root/.local/share/tldr/pages/common/`)
|
||||
- Analysis-specific tools not in standard TLDR
|
||||
- Follows standard TLDR format
|
||||
- Quick reference examples
|
||||
|
||||
### 5. **Unified fhelp Command**
|
||||
- Single interface to all help systems
|
||||
- Clear command structure with examples
|
||||
- Workflow-based help (PDF, malware analysis)
|
||||
- Offline capability verification
|
||||
|
||||
## 📋 Usage Examples
|
||||
|
||||
### Find Tools
|
||||
```bash
|
||||
fhelp tools pdf # Find PDF analysis tools
|
||||
fhelp tools malware # Find malware analysis tools
|
||||
fhelp tools --interactive # Interactive tool browser
|
||||
```
|
||||
|
||||
### Get Command Examples
|
||||
```bash
|
||||
fhelp cheat pdf # PDF analysis workflow
|
||||
fhelp cheat pdftk # pdftk-specific examples
|
||||
fhelp quick tar # Quick tar examples
|
||||
fhelp examples # Interactive example browser
|
||||
```
|
||||
|
||||
### Analysis Workflows
|
||||
```bash
|
||||
fhelp pdf # Step-by-step PDF analysis
|
||||
fhelp malware # Step-by-step malware analysis
|
||||
```
|
||||
|
||||
### Verification
|
||||
```bash
|
||||
fhelp --offline # Check all help systems work offline
|
||||
fhelp --all # Complete overview of capabilities
|
||||
```
|
||||
|
||||
## 🎯 Key Improvements for File Analysis
|
||||
|
||||
### PDF Security Analysis
|
||||
- **Flattening**: `pdftk suspicious.pdf output safe.pdf flatten`
|
||||
- **Password Removal**: `qpdf --password=PASSWORD --decrypt encrypted.pdf decrypted.pdf`
|
||||
- **JavaScript Detection**: `strings document.pdf | grep -i "javascript"`
|
||||
- **Safe Preview**: `convert document.pdf[0] preview.png`
|
||||
|
||||
### Malware Analysis Workflow
|
||||
- **Capability Detection**: `capa malware.exe`
|
||||
- **JavaScript Sandbox**: `box-js --output-dir=/tmp suspicious.js`
|
||||
- **Office Document Analysis**: `oledump.py document.doc`
|
||||
- **Binary Analysis**: `binwalk malware.bin`
|
||||
|
||||
### Data Analysis
|
||||
- **Interactive Exploration**: `vd data.csv`
|
||||
- **URL Analysis**: `unfurl http://suspicious.com/path`
|
||||
|
||||
## 🔧 Testing
|
||||
|
||||
Run comprehensive tests with:
|
||||
```bash
|
||||
./test-help-system.sh
|
||||
```
|
||||
|
||||
This validates:
|
||||
- All help commands work correctly
|
||||
- Tool availability matches documentation
|
||||
- File structure is correct
|
||||
- Integration between help systems works
|
||||
|
||||
## 📦 Build Instructions
|
||||
|
||||
The enhanced help system is automatically included when building:
|
||||
|
||||
```bash
|
||||
docker build -f Dockerfile.remnux -t tabledevil/file-analysis:enhanced .
|
||||
```
|
||||
|
||||
## 🎉 Benefits
|
||||
|
||||
1. **No More Missing Tools**: Only lists tools that actually exist with correct names
|
||||
2. **Practical Examples**: Real-world analysis commands, not just descriptions
|
||||
3. **Workflow Guidance**: Step-by-step analysis procedures
|
||||
4. **Offline Capable**: Works completely offline with comprehensive documentation
|
||||
5. **Multiple Interfaces**: Choose between fhelp, navi, cheat, tldr based on preference
|
||||
6. **Security Focus**: Emphasizes safe analysis practices (PDF flattening, sandboxing)
|
||||
|
||||
## 📱 Quick Reference Card
|
||||
|
||||
| Command | Purpose | Example |
|
||||
|---------|---------|---------|
|
||||
| `fhelp` | Main help system | `fhelp tools pdf` |
|
||||
| `fhelp cheat <tool>` | Tool examples | `fhelp cheat pdftk` |
|
||||
| `fhelp pdf` | PDF workflow | Shows step-by-step PDF analysis |
|
||||
| `fhelp malware` | Malware workflow | Shows malware analysis steps |
|
||||
| `navi` | Interactive browser | Browse all examples with fzf |
|
||||
| `find-tool <term>` | Tool search | `find-tool javascript` |
|
||||
| `tldr <tool>` | Quick reference | `tldr tar` |
|
||||
| `cheat <tool>` | Detailed examples | `cheat 7z` |
|
||||
|
||||
This enhanced system provides comprehensive, accurate, offline-capable help for file analysis workflows while leveraging existing community resources and adding specialized analysis examples.
|
||||
Reference in New Issue
Block a user