From 27760b0bf1c5d8e172b62daebfcc9d7977387eba Mon Sep 17 00:00:00 2001 From: tobias Date: Sun, 15 Feb 2026 18:12:53 +0100 Subject: [PATCH] visidata: document config and plugins --- config/visidata/README.md | 124 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100644 config/visidata/README.md diff --git a/config/visidata/README.md b/config/visidata/README.md new file mode 100644 index 0000000..b96f7cc --- /dev/null +++ b/config/visidata/README.md @@ -0,0 +1,124 @@ +# VisiData Config + Plugins + +This folder contains a VisiData `config.py` (symlinked from `visidatarc`) plus a small set of local plugins under `plugins/`. + +## Install + +The installer links (or copies) the config and plugins into VisiData’s per-user directory. + +```bash +./install.sh --link # default, symlinks into place +./install.sh --copy # copies into place +./install.sh --deps # installs optional Python deps into $VD_DIR/plugins-deps +``` + +On VisiData 3.3, `$VD_DIR` defaults to: +- macOS: `~/Library/Preferences/visidata` +- Linux: `${XDG_CONFIG_HOME:-~/.config}/visidata` + +## Plugins + +Plugins are installed into `$VD_DIR/plugins/` and imported via the top-level `plugins` package. + +### `plugins/hidecol.py` + +Adds a command to hide columns that are empty or constant across all rows. + +- Command: `tke-hidecol` +- Menu: `Column -> Hide -> empty/superfluous columns` + +### `plugins/iptype.py` + +Adds a custom IP datatype that supports: +- IPv4 + IPv6 addresses +- CIDR networks (e.g. `192.168.7.0/24`) +- Correct sorting (numeric, by version) +- Membership test operator: `ip * net` (and `net * ip`) +- Normalized lookup/enrichment properties, accessible as attributes in expressions + +#### Type + Command + +- Type converter: `ip(...)` +- Type name: `IP` +- Command: `type-ip` (sets `cursorCol.type=ip`) + +#### Operations + +Membership test: +- `ipcol * "192.168.7.0/24"` -> `True`/`False` +- `"192.168.7.0/24" * ipcol` -> `True`/`False` + +#### Attributes (on `IP` typed cells) + +Lookup objects expose both normalized fields and raw response data: + +- `ipcol.ipinfo.country` +- `ipcol.ipinfo.data.` +- `ipcol.asn.asn`, `ipcol.asn.name`, `ipcol.asn.country` +- `ipcol.asn.data.` +- `ipcol.vt.verdict` (e.g. `"3/94"`), `ipcol.vt.malicious`, `ipcol.vt.total`, `ipcol.vt.category` (alias: `ipcol.vt.type`) +- `ipcol.vt.data.` +- `ipcol.geo.*` (best-available geo: prefers MaxMind mmdb, else free HTTP providers) +- `ipcol.maxmind.*` (offline-only MaxMind lookup; empty if no mmdb) + +#### Caching + +All lookup providers cache results in a local sqlite+pickle DB (default `~/.visidata_cache.db`). + +#### Lookup Providers + Keys + +Options (set in `config.py` / `visidatarc`): +- `options.tke_cache_db_path="~/.visidata_cache.db"` +- `options.tke_lookup_cache_ttl=86400` +- `options.tke_lookup_error_ttl=300` +- `options.tke_lookup_timeout=10` +- `options.tke_ipinfo_token="..."` (optional; ipinfo can work without it) +- `options.tke_ipapi_key="..."` (optional) +- `options.tke_vt_api_key="..."` (required for VT lookups unless using `~/.virustotal_api_key`) +- `options.tke_maxmind_mmdb_path="/path/to/GeoLite2-City.mmdb"` (optional) + +Env var equivalents: +- `IPINFO_TOKEN`, `IPAPI_KEY` +- `VT_API_KEY` or `VIRUSTOTAL_API_KEY` (also supports `~/.virustotal_api_key`) +- `MAXMIND_MMDB_PATH` or `GEOIP_MMDB_PATH` + +MaxMind (offline “free” GeoLite2) support: +- Place a `GeoLite2-City.mmdb` / `GeoLite2-Country.mmdb` file in `$VD_DIR/`, or set `options.tke_maxmind_mmdb_path`. + +### `plugins/iplib.py` + +Pure-Python library used by `iptype.py` for: +- Normalized info classes (`IPInfo`, `ASNInfo`, `VTInfo`, `GeoInfo`) +- `JSONNode` wrapper (`.data.`) for safe attribute-style access into raw dict/list JSON +- Parsing/normalization helpers for each provider’s response shape + +This file intentionally does **not** import VisiData so it can be validated outside the VisiData runtime. + +## Config: `visidatarc` + +This repo’s `visidatarc` is intended to be installed as VisiData’s `config.py`: +- `$VD_DIR/config.py` (VisiData 3.3 default) +- and also `~/.visidatarc` as a legacy fallback + +It currently contains: +- display/date format options +- a sqlite+pickle caching decorator and a set of general-purpose helpers (aggregators, timestamp parsing, “dirty” JSON parsing, etc) + +### Are the `visidatarc` functions superseded? + +Partially: +- The **IP-centric lookups and normalized attribute access** are now primarily handled by `plugins/iptype.py` on typed values (e.g. `ipcol.geo.country_code`). +- Many other helpers in `visidatarc` (aggregators like `avgdiff`, parsing/time conversion helpers, etc.) are still independent and useful. + +### Keeping old + new side-by-side (without duplicating code) + +Yes. The cleanest pattern in VisiData is: +1. Put shared logic into a module under `plugins/` (so it’s on `sys.path` via `$VD_DIR`). +2. In `visidatarc`, import and expose thin wrappers (or just import the module and use `module.func(...)` in expressions). + +Concretely: +- `plugins/iplib.py` already holds parsing/normalization shared by the IP type. +- If you have legacy functions in `visidatarc` that overlap with the new IP lookups, refactor those functions into a shared module (e.g. `plugins/lookups.py`) and have both `visidatarc` and `plugins/iptype.py` call into it. + +This keeps backward-compatible names available while ensuring caching/auth/provider behavior is implemented in one place. +