visidata: document config and plugins
This commit is contained in:
tobias
124
config/visidata/README.md
Normal file
124
config/visidata/README.md
Normal file
@@ -0,0 +1,124 @@
|
|||||||
|
# VisiData Config + Plugins
|
||||||
|
|
||||||
|
This folder contains a VisiData `config.py` (symlinked from `visidatarc`) plus a small set of local plugins under `plugins/`.
|
||||||
|
|
||||||
|
## Install
|
||||||
|
|
||||||
|
The installer links (or copies) the config and plugins into VisiData’s per-user directory.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./install.sh --link # default, symlinks into place
|
||||||
|
./install.sh --copy # copies into place
|
||||||
|
./install.sh --deps # installs optional Python deps into $VD_DIR/plugins-deps
|
||||||
|
```
|
||||||
|
|
||||||
|
On VisiData 3.3, `$VD_DIR` defaults to:
|
||||||
|
- macOS: `~/Library/Preferences/visidata`
|
||||||
|
- Linux: `${XDG_CONFIG_HOME:-~/.config}/visidata`
|
||||||
|
|
||||||
|
## Plugins
|
||||||
|
|
||||||
|
Plugins are installed into `$VD_DIR/plugins/` and imported via the top-level `plugins` package.
|
||||||
|
|
||||||
|
### `plugins/hidecol.py`
|
||||||
|
|
||||||
|
Adds a command to hide columns that are empty or constant across all rows.
|
||||||
|
|
||||||
|
- Command: `tke-hidecol`
|
||||||
|
- Menu: `Column -> Hide -> empty/superfluous columns`
|
||||||
|
|
||||||
|
### `plugins/iptype.py`
|
||||||
|
|
||||||
|
Adds a custom IP datatype that supports:
|
||||||
|
- IPv4 + IPv6 addresses
|
||||||
|
- CIDR networks (e.g. `192.168.7.0/24`)
|
||||||
|
- Correct sorting (numeric, by version)
|
||||||
|
- Membership test operator: `ip * net` (and `net * ip`)
|
||||||
|
- Normalized lookup/enrichment properties, accessible as attributes in expressions
|
||||||
|
|
||||||
|
#### Type + Command
|
||||||
|
|
||||||
|
- Type converter: `ip(...)`
|
||||||
|
- Type name: `IP`
|
||||||
|
- Command: `type-ip` (sets `cursorCol.type=ip`)
|
||||||
|
|
||||||
|
#### Operations
|
||||||
|
|
||||||
|
Membership test:
|
||||||
|
- `ipcol * "192.168.7.0/24"` -> `True`/`False`
|
||||||
|
- `"192.168.7.0/24" * ipcol` -> `True`/`False`
|
||||||
|
|
||||||
|
#### Attributes (on `IP` typed cells)
|
||||||
|
|
||||||
|
Lookup objects expose both normalized fields and raw response data:
|
||||||
|
|
||||||
|
- `ipcol.ipinfo.country`
|
||||||
|
- `ipcol.ipinfo.data.<any_json_field>`
|
||||||
|
- `ipcol.asn.asn`, `ipcol.asn.name`, `ipcol.asn.country`
|
||||||
|
- `ipcol.asn.data.<any_json_field>`
|
||||||
|
- `ipcol.vt.verdict` (e.g. `"3/94"`), `ipcol.vt.malicious`, `ipcol.vt.total`, `ipcol.vt.category` (alias: `ipcol.vt.type`)
|
||||||
|
- `ipcol.vt.data.<any_json_field>`
|
||||||
|
- `ipcol.geo.*` (best-available geo: prefers MaxMind mmdb, else free HTTP providers)
|
||||||
|
- `ipcol.maxmind.*` (offline-only MaxMind lookup; empty if no mmdb)
|
||||||
|
|
||||||
|
#### Caching
|
||||||
|
|
||||||
|
All lookup providers cache results in a local sqlite+pickle DB (default `~/.visidata_cache.db`).
|
||||||
|
|
||||||
|
#### Lookup Providers + Keys
|
||||||
|
|
||||||
|
Options (set in `config.py` / `visidatarc`):
|
||||||
|
- `options.tke_cache_db_path="~/.visidata_cache.db"`
|
||||||
|
- `options.tke_lookup_cache_ttl=86400`
|
||||||
|
- `options.tke_lookup_error_ttl=300`
|
||||||
|
- `options.tke_lookup_timeout=10`
|
||||||
|
- `options.tke_ipinfo_token="..."` (optional; ipinfo can work without it)
|
||||||
|
- `options.tke_ipapi_key="..."` (optional)
|
||||||
|
- `options.tke_vt_api_key="..."` (required for VT lookups unless using `~/.virustotal_api_key`)
|
||||||
|
- `options.tke_maxmind_mmdb_path="/path/to/GeoLite2-City.mmdb"` (optional)
|
||||||
|
|
||||||
|
Env var equivalents:
|
||||||
|
- `IPINFO_TOKEN`, `IPAPI_KEY`
|
||||||
|
- `VT_API_KEY` or `VIRUSTOTAL_API_KEY` (also supports `~/.virustotal_api_key`)
|
||||||
|
- `MAXMIND_MMDB_PATH` or `GEOIP_MMDB_PATH`
|
||||||
|
|
||||||
|
MaxMind (offline “free” GeoLite2) support:
|
||||||
|
- Place a `GeoLite2-City.mmdb` / `GeoLite2-Country.mmdb` file in `$VD_DIR/`, or set `options.tke_maxmind_mmdb_path`.
|
||||||
|
|
||||||
|
### `plugins/iplib.py`
|
||||||
|
|
||||||
|
Pure-Python library used by `iptype.py` for:
|
||||||
|
- Normalized info classes (`IPInfo`, `ASNInfo`, `VTInfo`, `GeoInfo`)
|
||||||
|
- `JSONNode` wrapper (`.data.<field>`) for safe attribute-style access into raw dict/list JSON
|
||||||
|
- Parsing/normalization helpers for each provider’s response shape
|
||||||
|
|
||||||
|
This file intentionally does **not** import VisiData so it can be validated outside the VisiData runtime.
|
||||||
|
|
||||||
|
## Config: `visidatarc`
|
||||||
|
|
||||||
|
This repo’s `visidatarc` is intended to be installed as VisiData’s `config.py`:
|
||||||
|
- `$VD_DIR/config.py` (VisiData 3.3 default)
|
||||||
|
- and also `~/.visidatarc` as a legacy fallback
|
||||||
|
|
||||||
|
It currently contains:
|
||||||
|
- display/date format options
|
||||||
|
- a sqlite+pickle caching decorator and a set of general-purpose helpers (aggregators, timestamp parsing, “dirty” JSON parsing, etc)
|
||||||
|
|
||||||
|
### Are the `visidatarc` functions superseded?
|
||||||
|
|
||||||
|
Partially:
|
||||||
|
- The **IP-centric lookups and normalized attribute access** are now primarily handled by `plugins/iptype.py` on typed values (e.g. `ipcol.geo.country_code`).
|
||||||
|
- Many other helpers in `visidatarc` (aggregators like `avgdiff`, parsing/time conversion helpers, etc.) are still independent and useful.
|
||||||
|
|
||||||
|
### Keeping old + new side-by-side (without duplicating code)
|
||||||
|
|
||||||
|
Yes. The cleanest pattern in VisiData is:
|
||||||
|
1. Put shared logic into a module under `plugins/` (so it’s on `sys.path` via `$VD_DIR`).
|
||||||
|
2. In `visidatarc`, import and expose thin wrappers (or just import the module and use `module.func(...)` in expressions).
|
||||||
|
|
||||||
|
Concretely:
|
||||||
|
- `plugins/iplib.py` already holds parsing/normalization shared by the IP type.
|
||||||
|
- If you have legacy functions in `visidatarc` that overlap with the new IP lookups, refactor those functions into a shared module (e.g. `plugins/lookups.py`) and have both `visidatarc` and `plugins/iptype.py` call into it.
|
||||||
|
|
||||||
|
This keeps backward-compatible names available while ensuring caching/auth/provider behavior is implemented in one place.
|
||||||
|
|
||||||
Reference in New Issue
Block a user