Add FOR610 tool/workflow knowledge base and data pipeline

Build comprehensive malware analysis knowledge base from 3 sources:
- SANS FOR610 course: 120 tools, 47 labs, 15 workflows, 27 recipes
- REMnux salt-states: 340 packages parsed from GitHub
- REMnux docs: 280+ tools scraped from docs.remnux.org

Master inventory merges all sources into 447 tools with help tiers
(rich/standard/basic). Pipeline generates: tools.db (397 entries),
397 cheatsheets with multi-tool recipes, 15 workflow guides, 224
TLDR pages, and coverage reports.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
tobias
2026-03-28 17:38:15 +01:00
parent 06ebb09ab0
commit f3ccc09c3d
663 changed files with 36339 additions and 1 deletions
+30 -1
View File
@@ -1,4 +1,4 @@
.PHONY: help build-upstream build-scratch build-kali build-all test clean push
.PHONY: help build-upstream build-scratch build-kali build-all test clean push generate-data generate-master coverage-report
# Default target
help:
@@ -16,6 +16,9 @@ help:
@echo " shell Interactive shell (REMnux build)"
@echo " shell-scratch Interactive shell (scratch build)"
@echo " coverage Check help coverage"
@echo " generate-data Convert FOR610 YAML to JSON"
@echo " generate-master Build master inventory and all help artifacts"
@echo " coverage-report Generate tool coverage gap report"
@echo ""
# Build targets
@@ -63,3 +66,29 @@ shell:
coverage:
@echo "Checking help coverage..."
@docker run --rm tabledevil/file-analysis:latest /usr/local/bin/check-help-coverage.sh || true
# Generate JSON from FOR610 YAML knowledge base
generate-data:
@echo "Generating JSON from FOR610 YAML files..."
@mkdir -p data/generated
@for f in data/for610/*.yaml; do \
name=$$(basename "$$f" .yaml); \
python3 -c "import yaml,json; json.dump(yaml.safe_load(open('$$f')),open('data/generated/$$name.json','w'),indent=2)"; \
echo "$$name.json"; \
done
@echo "✓ JSON files generated in data/generated/"
# Build master inventory from all 3 sources and generate all help artifacts
generate-master: generate-data
@echo "Building master tool inventory..."
python3 scripts/parse-salt-states.py
python3 scripts/scrape-remnux-docs.py
python3 scripts/build-master-inventory.py
python3 scripts/generate-help-artifacts.py
python3 scripts/generate-coverage-report.py
@echo "✓ Master inventory and all artifacts generated"
# Generate coverage report only (requires tools-master.yaml to exist)
coverage-report:
python3 scripts/generate-coverage-report.py
@echo "Report: data/generated/coverage-report.md"