- Reorganize documentation: moved old docs to docs/ directory - Add comprehensive README.md with build options and usage guide - Add detailed CONTRIBUTING.md with help content management guide - Create Makefile for automated building and testing - Add Dockerfile.scratch for building from Ubuntu 20.04 base - Enhance all Dockerfiles with PowerShell + PSScriptAnalyzer - Add modern shells: zsh (with plugins) and fish (with config) - Add modern CLI tools: fd-find, ripgrep, fzf - Create comprehensive help system with cheat/TLDR/fish completions - Add helper scripts for help content management and coverage checking - Fix Dockerfile.remnux script references - Support three build variants: upstream (REMnux), scratch (Ubuntu), kali Build options: - make build-upstream: Fast, uses REMnux upstream (recommended) - make build-scratch: Full control, builds from Ubuntu 20.04 - make build-kali: Legacy Kali Linux base Features: - PowerShell with PSScriptAnalyzer module - Modern shells (zsh, fish) with custom configurations - Enhanced help system (cheat sheets, TLDR pages, fish completions) - Help coverage checking and bulk import tools - Comprehensive documentation for users and contributors
615 lines
13 KiB
Markdown
615 lines
13 KiB
Markdown
# Contributing to File Analysis Container
|
|
|
|
Thank you for your interest in improving the File Analysis Container! This guide will help you add and modify help content, cheat sheets, TLDR pages, and shell configurations.
|
|
|
|
## 📋 Table of Contents
|
|
|
|
- [Getting Started](#getting-started)
|
|
- [Adding Cheat Sheets](#adding-cheat-sheets)
|
|
- [Adding TLDR Pages](#adding-tldr-pages)
|
|
- [Adding Fish Shell Completions](#adding-fish-shell-completions)
|
|
- [Importing Bulk Cheatsheets](#importing-bulk-cheatsheets)
|
|
- [Checking Help Coverage](#checking-help-coverage)
|
|
- [Adding New Tools](#adding-new-tools)
|
|
- [Testing Your Changes](#testing-your-changes)
|
|
|
|
## 🚀 Getting Started
|
|
|
|
### Prerequisites
|
|
|
|
```bash
|
|
# Clone the repository
|
|
git clone https://github.com/tabledevil/docker_file_analysis.git
|
|
cd docker_file_analysis
|
|
|
|
# Build the container
|
|
make build-upstream
|
|
```
|
|
|
|
### Directory Structure
|
|
|
|
```
|
|
docker_file_analysis/
|
|
├── scripts/
|
|
│ ├── fhelp # Main help system
|
|
│ ├── add-tool-cheats.sh # Add default cheat sheets
|
|
│ ├── create-offline-help-system.sh # Initialize help database
|
|
│ ├── import-remnux-cheatsheets.sh # Import from markdown
|
|
│ └── convert-remnux-cheats.py # Markdown converter
|
|
├── cheatsheets/
|
|
│ └── *.cheat # Custom cheat sheets
|
|
└── files/
|
|
├── command_help # Detailed examples
|
|
└── README # Welcome message
|
|
```
|
|
|
|
## 📝 Adding Cheat Sheets
|
|
|
|
Cheat sheets provide quick command examples and common usage patterns.
|
|
|
|
### Format
|
|
|
|
Cheat sheets use a simple markdown-like syntax:
|
|
|
|
```bash
|
|
# tool-name - Short description
|
|
|
|
# Example description
|
|
tool-name basic_command
|
|
|
|
# Another example with options
|
|
tool-name --option argument
|
|
|
|
# More complex example
|
|
tool-name --verbose --output result.txt input.txt
|
|
```
|
|
|
|
### Location
|
|
|
|
- **Personal cheat sheets:** `/opt/cheatsheets/personal/`
|
|
- **Community cheat sheets:** `/opt/cheatsheets/community/`
|
|
|
|
### Creating a New Cheat Sheet
|
|
|
|
**Inside the container:**
|
|
|
|
```bash
|
|
# Create the cheat sheet
|
|
cat > /opt/cheatsheets/personal/mytool << 'EOF'
|
|
# mytool - Analysis tool for files
|
|
|
|
# Basic file analysis
|
|
mytool sample.bin
|
|
|
|
# Verbose output
|
|
mytool -v sample.bin
|
|
|
|
# Export results to JSON
|
|
mytool --format json --output results.json sample.bin
|
|
|
|
# Scan directory recursively
|
|
mytool -r /data/samples/
|
|
|
|
# Common analysis workflow
|
|
mytool --extract-strings sample.bin | grep -i "http"
|
|
EOF
|
|
|
|
# Test it
|
|
fhelp cheat mytool
|
|
```
|
|
|
|
**Outside the container (for permanent addition):**
|
|
|
|
```bash
|
|
# Add to repository
|
|
cat > cheatsheets/mytool.cheat << 'EOF'
|
|
# mytool - Analysis tool for files
|
|
...
|
|
EOF
|
|
|
|
# Rebuild container
|
|
make build-upstream
|
|
```
|
|
|
|
### Cheat Sheet Best Practices
|
|
|
|
1. **Start with tool name and brief description**
|
|
2. **Order examples from simple to complex**
|
|
3. **Include common workflow patterns**
|
|
4. **Add security-relevant examples**
|
|
5. **Keep examples practical and concise**
|
|
|
|
### Example: PDF Analysis Cheat Sheet
|
|
|
|
```bash
|
|
# pdfid.py - Quick PDF structure analysis
|
|
|
|
# Basic PDF analysis
|
|
pdfid.py document.pdf
|
|
|
|
# Scan all PDFs in directory
|
|
pdfid.py *.pdf
|
|
|
|
# Look for suspicious elements
|
|
pdfid.py -a document.pdf
|
|
|
|
# Export to JSON
|
|
pdfid.py -o json document.pdf
|
|
|
|
# Detect common malicious patterns
|
|
pdfid.py document.pdf | grep -i "javascript\|openaction\|aa\|launch"
|
|
```
|
|
|
|
## 📖 Adding TLDR Pages
|
|
|
|
TLDR pages are simplified man pages focusing on practical examples.
|
|
|
|
### Format
|
|
|
|
TLDR pages follow a specific structure:
|
|
|
|
```markdown
|
|
# tool-name
|
|
|
|
> Brief description of what the tool does.
|
|
> More information: <https://tool-website.com>.
|
|
|
|
- Brief description of example:
|
|
|
|
`tool-name argument`
|
|
|
|
- Another example with options:
|
|
|
|
`tool-name --option {{value}}`
|
|
|
|
- Complex example:
|
|
|
|
`tool-name --input {{input.txt}} --output {{output.txt}}`
|
|
```
|
|
|
|
### Location
|
|
|
|
- **TLDR cache:** `~/.local/share/tldr/`
|
|
- **Pages:** `~/.local/share/tldr/pages/common/`
|
|
|
|
### Creating a TLDR Page
|
|
|
|
```bash
|
|
# Create directory if needed
|
|
mkdir -p ~/.local/share/tldr/pages/common
|
|
|
|
# Create TLDR page
|
|
cat > ~/.local/share/tldr/pages/common/mytool.md << 'EOF'
|
|
# mytool
|
|
|
|
> Analysis tool for examining potentially malicious files.
|
|
> Part of the file analysis toolkit.
|
|
> More information: <https://github.com/example/mytool>.
|
|
|
|
- Analyze a single file:
|
|
|
|
`mytool {{path/to/file}}`
|
|
|
|
- Analyze with verbose output:
|
|
|
|
`mytool --verbose {{path/to/file}}`
|
|
|
|
- Export analysis results:
|
|
|
|
`mytool --output {{results.json}} {{path/to/file}}`
|
|
|
|
- Scan directory recursively:
|
|
|
|
`mytool --recursive {{path/to/directory}}`
|
|
EOF
|
|
|
|
# Test it
|
|
tldr mytool
|
|
# OR
|
|
fhelp tldr mytool
|
|
```
|
|
|
|
### TLDR Best Practices
|
|
|
|
1. **Use placeholders** like `{{filename}}` for user input
|
|
2. **Keep descriptions brief** (one line)
|
|
3. **Include URL** to official documentation
|
|
4. **Start with simplest examples**
|
|
5. **Cover most common use cases** (4-8 examples)
|
|
|
|
## 🐚 Adding Fish Shell Completions
|
|
|
|
Fish completions provide smart command-line autocompletion.
|
|
|
|
### Format
|
|
|
|
Fish completions use a declarative syntax:
|
|
|
|
```fish
|
|
# mytool completions
|
|
complete -c mytool -s v -l verbose -d "Enable verbose output"
|
|
complete -c mytool -s o -l output -d "Output file" -r
|
|
complete -c mytool -s r -l recursive -d "Scan recursively"
|
|
complete -c mytool -s h -l help -d "Show help message"
|
|
```
|
|
|
|
### Location
|
|
|
|
Fish completions go in: `~/.config/fish/completions/`
|
|
|
|
### Creating Fish Completions
|
|
|
|
```bash
|
|
# Create directory
|
|
mkdir -p ~/.config/fish/completions
|
|
|
|
# Create completions file
|
|
cat > ~/.config/fish/completions/mytool.fish << 'EOF'
|
|
# mytool - File analysis tool completions
|
|
|
|
# Basic options
|
|
complete -c mytool -s v -l verbose -d "Verbose output"
|
|
complete -c mytool -s h -l help -d "Show help"
|
|
complete -c mytool -s V -l version -d "Show version"
|
|
|
|
# File/directory arguments
|
|
complete -c mytool -s o -l output -d "Output file" -r -F
|
|
complete -c mytool -s i -l input -d "Input file" -r -F
|
|
|
|
# Analysis options
|
|
complete -c mytool -l format -d "Output format" -r -a "json xml text"
|
|
complete -c mytool -l extract -d "Extract embedded files"
|
|
complete -c mytool -s r -l recursive -d "Recursive scan"
|
|
EOF
|
|
|
|
# Test it (restart fish or source completions)
|
|
fish
|
|
mytool --<TAB>
|
|
```
|
|
|
|
### Fish Completion Best Practices
|
|
|
|
1. **Complete both short (-v) and long (--verbose) options**
|
|
2. **Add descriptions** with `-d "description"`
|
|
3. **Use `-r`** for options requiring arguments
|
|
4. **Use `-F`** for file/path completion
|
|
5. **Use `-a`** for fixed value choices
|
|
|
|
## 📦 Importing Bulk Cheatsheets
|
|
|
|
For adding many cheat sheets at once from a consolidated markdown file.
|
|
|
|
### Prepare Your Markdown
|
|
|
|
Create a consolidated cheatsheet file with this structure:
|
|
|
|
```markdown
|
|
## tool1
|
|
|
|
```bash
|
|
# Example 1
|
|
tool1 command
|
|
```
|
|
|
|
## tool2
|
|
|
|
```bash
|
|
# Example 2
|
|
tool2 --option value
|
|
```
|
|
```
|
|
|
|
### Import Process
|
|
|
|
```bash
|
|
# Inside the container:
|
|
|
|
# Place your markdown file in /data
|
|
# (mounted from host)
|
|
|
|
# Import cheatsheets
|
|
import-remnux-cheatsheets.sh /data/my-cheatsheets.md
|
|
|
|
# Verify import
|
|
fhelp cheat tool1
|
|
fhelp cheat tool2
|
|
```
|
|
|
|
### Converter Script
|
|
|
|
The `convert-remnux-cheats.py` script handles the conversion:
|
|
|
|
```bash
|
|
# Convert markdown to cheat/tldr format
|
|
convert-remnux-cheats.py input.md output_dir/
|
|
|
|
# This creates:
|
|
# output_dir/cheat/tool1
|
|
# output_dir/cheat/tool2
|
|
# output_dir/tldr/tool1.md
|
|
# output_dir/tldr/tool2.md
|
|
```
|
|
|
|
### Bulk Import Example
|
|
|
|
```bash
|
|
# Example workflow
|
|
cat > my-tools.md << 'EOF'
|
|
## exiftool
|
|
|
|
```bash
|
|
# Extract metadata
|
|
exiftool file.jpg
|
|
|
|
# Remove all metadata
|
|
exiftool -all= file.jpg
|
|
```
|
|
|
|
## binwalk
|
|
|
|
```bash
|
|
# Scan for embedded files
|
|
binwalk firmware.bin
|
|
|
|
# Extract files
|
|
binwalk -e firmware.bin
|
|
```
|
|
EOF
|
|
|
|
# Import
|
|
import-remnux-cheatsheets.sh my-tools.md
|
|
|
|
# Verify
|
|
fhelp cheat exiftool
|
|
fhelp cheat binwalk
|
|
```
|
|
|
|
## 🔍 Checking Help Coverage
|
|
|
|
Use the coverage checker to find tools missing documentation.
|
|
|
|
### Run Coverage Check
|
|
|
|
```bash
|
|
# Inside container:
|
|
check-help-coverage.sh
|
|
|
|
# Output shows:
|
|
# ✓ Tools with both cheat and TLDR
|
|
# ⚠ Tools with only cheat
|
|
# ⚠ Tools with only TLDR
|
|
# ✗ Tools with no help
|
|
```
|
|
|
|
### Example Output
|
|
|
|
```
|
|
Checking help coverage for analysis tools...
|
|
|
|
✓ pdfid [cheat] [tldr]
|
|
✓ pdf-parser [cheat] [tldr]
|
|
⚠ oledump [cheat] [ ]
|
|
⚠ rtfdump [ ] [tldr]
|
|
✗ newtool [ ] [ ]
|
|
|
|
Summary:
|
|
Total tools: 5
|
|
Complete: 2 (40%)
|
|
Partial: 2 (40%)
|
|
Missing: 1 (20%)
|
|
```
|
|
|
|
### Adding Missing Help
|
|
|
|
Based on coverage report, add missing help content:
|
|
|
|
```bash
|
|
# Tool with no help
|
|
cat > /opt/cheatsheets/personal/newtool << 'EOF'
|
|
# newtool - Description
|
|
# Basic usage
|
|
newtool file
|
|
EOF
|
|
|
|
# Tool missing TLDR
|
|
mkdir -p ~/.local/share/tldr/pages/common
|
|
cat > ~/.local/share/tldr/pages/common/oledump.md << 'EOF'
|
|
# oledump
|
|
> Analyze OLE files
|
|
...
|
|
EOF
|
|
|
|
# Verify
|
|
check-help-coverage.sh
|
|
```
|
|
|
|
## 🔧 Adding New Tools
|
|
|
|
### 1. Add to Dockerfile
|
|
|
|
```dockerfile
|
|
# Install via apt
|
|
RUN apt-get update && apt-get install -y newtool
|
|
|
|
# OR install via pip
|
|
RUN pip install newtool
|
|
|
|
# OR install from source
|
|
RUN git clone https://github.com/author/newtool /opt/newtool \
|
|
&& cd /opt/newtool \
|
|
&& make install
|
|
```
|
|
|
|
### 2. Add Help Content
|
|
|
|
```bash
|
|
# Cheat sheet
|
|
cat > cheatsheets/newtool.cheat << 'EOF'
|
|
# newtool - Tool description
|
|
# Basic usage
|
|
newtool file.bin
|
|
EOF
|
|
|
|
# TLDR page
|
|
cat > tldr/newtool.md << 'EOF'
|
|
# newtool
|
|
> Tool description
|
|
- Basic usage:
|
|
`newtool {{file}}`
|
|
EOF
|
|
```
|
|
|
|
### 3. Add to Help System Script
|
|
|
|
Edit `scripts/add-tool-cheats.sh`:
|
|
|
|
```bash
|
|
# Add newtool cheat sheet
|
|
cat > "$CHEAT_DIR/newtool" << 'EOF'
|
|
# newtool - Tool description
|
|
# Basic usage
|
|
newtool file.bin
|
|
EOF
|
|
```
|
|
|
|
### 4. Rebuild and Test
|
|
|
|
```bash
|
|
# Rebuild container
|
|
make build-upstream
|
|
|
|
# Test
|
|
docker run --rm tabledevil/file-analysis:remnux newtool --version
|
|
docker run --rm tabledevil/file-analysis:remnux fhelp cheat newtool
|
|
```
|
|
|
|
## 🧪 Testing Your Changes
|
|
|
|
### Test Locally
|
|
|
|
```bash
|
|
# Build container
|
|
make build-upstream
|
|
|
|
# Run interactive session
|
|
docker run -it --rm tabledevil/file-analysis:remnux
|
|
|
|
# Inside container, test help system
|
|
fhelp cheat yourtool
|
|
fhelp tldr yourtool
|
|
```
|
|
|
|
### Test Help System
|
|
|
|
```bash
|
|
# Test main help
|
|
docker run --rm tabledevil/file-analysis:remnux fhelp
|
|
|
|
# Test tool search
|
|
docker run --rm tabledevil/file-analysis:remnux fhelp tools pdf
|
|
|
|
# Test cheat access
|
|
docker run --rm tabledevil/file-analysis:remnux fhelp cheat pdfid
|
|
|
|
# Run coverage check
|
|
docker run --rm tabledevil/file-analysis:remnux check-help-coverage.sh
|
|
```
|
|
|
|
### Test Shells
|
|
|
|
```bash
|
|
# Test zsh
|
|
docker run --rm -it tabledevil/file-analysis:remnux zsh
|
|
|
|
# Test fish
|
|
docker run --rm -it tabledevil/file-analysis:remnux fish
|
|
|
|
# Test completions
|
|
docker run --rm -it tabledevil/file-analysis:remnux fish -c "complete -C'pdfid.py -'"
|
|
```
|
|
|
|
## 📤 Submitting Changes
|
|
|
|
### Create a Pull Request
|
|
|
|
1. **Fork the repository**
|
|
2. **Create a feature branch**
|
|
```bash
|
|
git checkout -b add-mytool-help
|
|
```
|
|
3. **Make your changes**
|
|
- Add cheat sheets
|
|
- Add TLDR pages
|
|
- Update scripts if needed
|
|
4. **Test your changes**
|
|
```bash
|
|
make build-upstream
|
|
make test
|
|
```
|
|
5. **Commit and push**
|
|
```bash
|
|
git add cheatsheets/mytool.cheat
|
|
git commit -m "Add cheat sheet for mytool"
|
|
git push origin add-mytool-help
|
|
```
|
|
6. **Open pull request** on GitHub
|
|
|
|
### Commit Message Guidelines
|
|
|
|
```
|
|
Add cheat sheet for mytool
|
|
|
|
- Include basic usage examples
|
|
- Add common workflow patterns
|
|
- Covers malware analysis use cases
|
|
```
|
|
|
|
## 🎯 Quick Reference
|
|
|
|
### Common Tasks
|
|
|
|
| Task | Command |
|
|
|------|---------|
|
|
| Add cheat sheet | `vim /opt/cheatsheets/personal/tool` |
|
|
| Add TLDR page | `vim ~/.local/share/tldr/pages/common/tool.md` |
|
|
| Test cheat | `fhelp cheat tool` |
|
|
| Test TLDR | `tldr tool` or `fhelp tldr tool` |
|
|
| Check coverage | `check-help-coverage.sh` |
|
|
| Import bulk | `import-remnux-cheatsheets.sh file.md` |
|
|
|
|
### File Locations
|
|
|
|
| Content Type | Location |
|
|
|--------------|----------|
|
|
| Cheat sheets | `/opt/cheatsheets/personal/` |
|
|
| TLDR pages | `~/.local/share/tldr/pages/common/` |
|
|
| Fish completions | `~/.config/fish/completions/` |
|
|
| Help script | `/usr/local/bin/fhelp` |
|
|
|
|
## 💡 Tips and Best Practices
|
|
|
|
1. **Test in container** - Always test help content inside the container
|
|
2. **Keep examples practical** - Focus on real analysis workflows
|
|
3. **Use consistent formatting** - Follow existing cheat sheet styles
|
|
4. **Add context** - Include brief descriptions for each example
|
|
5. **Cover common cases** - 80% of use cases with 20% of examples
|
|
6. **Security focus** - Emphasize malware analysis patterns
|
|
7. **Cross-reference** - Link related tools in descriptions
|
|
|
|
## 🆘 Getting Help
|
|
|
|
- **Issues**: [GitHub Issues](https://github.com/tabledevil/docker_file_analysis/issues)
|
|
- **Discussions**: [GitHub Discussions](https://github.com/tabledevil/docker_file_analysis/discussions)
|
|
- **Documentation**: See `docs/` directory for archived documentation
|
|
|
|
## 📚 Additional Resources
|
|
|
|
- [cheat syntax guide](https://github.com/cheat/cheat)
|
|
- [TLDR pages format](https://github.com/tldr-pages/tldr/blob/main/CONTRIBUTING.md)
|
|
- [Fish shell completions](https://fishshell.com/docs/current/completions.html)
|
|
- [REMnux documentation](https://docs.remnux.org/)
|
|
|
|
---
|
|
|
|
Thank you for contributing to the File Analysis Container project! 🎉
|