visidata: document config and plugins
This commit is contained in:
tobias
124
config/visidata/README.md
Normal file
124
config/visidata/README.md
Normal file
@@ -0,0 +1,124 @@
|
||||
# VisiData Config + Plugins
|
||||
|
||||
This folder contains a VisiData `config.py` (symlinked from `visidatarc`) plus a small set of local plugins under `plugins/`.
|
||||
|
||||
## Install
|
||||
|
||||
The installer links (or copies) the config and plugins into VisiData’s per-user directory.
|
||||
|
||||
```bash
|
||||
./install.sh --link # default, symlinks into place
|
||||
./install.sh --copy # copies into place
|
||||
./install.sh --deps # installs optional Python deps into $VD_DIR/plugins-deps
|
||||
```
|
||||
|
||||
On VisiData 3.3, `$VD_DIR` defaults to:
|
||||
- macOS: `~/Library/Preferences/visidata`
|
||||
- Linux: `${XDG_CONFIG_HOME:-~/.config}/visidata`
|
||||
|
||||
## Plugins
|
||||
|
||||
Plugins are installed into `$VD_DIR/plugins/` and imported via the top-level `plugins` package.
|
||||
|
||||
### `plugins/hidecol.py`
|
||||
|
||||
Adds a command to hide columns that are empty or constant across all rows.
|
||||
|
||||
- Command: `tke-hidecol`
|
||||
- Menu: `Column -> Hide -> empty/superfluous columns`
|
||||
|
||||
### `plugins/iptype.py`
|
||||
|
||||
Adds a custom IP datatype that supports:
|
||||
- IPv4 + IPv6 addresses
|
||||
- CIDR networks (e.g. `192.168.7.0/24`)
|
||||
- Correct sorting (numeric, by version)
|
||||
- Membership test operator: `ip * net` (and `net * ip`)
|
||||
- Normalized lookup/enrichment properties, accessible as attributes in expressions
|
||||
|
||||
#### Type + Command
|
||||
|
||||
- Type converter: `ip(...)`
|
||||
- Type name: `IP`
|
||||
- Command: `type-ip` (sets `cursorCol.type=ip`)
|
||||
|
||||
#### Operations
|
||||
|
||||
Membership test:
|
||||
- `ipcol * "192.168.7.0/24"` -> `True`/`False`
|
||||
- `"192.168.7.0/24" * ipcol` -> `True`/`False`
|
||||
|
||||
#### Attributes (on `IP` typed cells)
|
||||
|
||||
Lookup objects expose both normalized fields and raw response data:
|
||||
|
||||
- `ipcol.ipinfo.country`
|
||||
- `ipcol.ipinfo.data.<any_json_field>`
|
||||
- `ipcol.asn.asn`, `ipcol.asn.name`, `ipcol.asn.country`
|
||||
- `ipcol.asn.data.<any_json_field>`
|
||||
- `ipcol.vt.verdict` (e.g. `"3/94"`), `ipcol.vt.malicious`, `ipcol.vt.total`, `ipcol.vt.category` (alias: `ipcol.vt.type`)
|
||||
- `ipcol.vt.data.<any_json_field>`
|
||||
- `ipcol.geo.*` (best-available geo: prefers MaxMind mmdb, else free HTTP providers)
|
||||
- `ipcol.maxmind.*` (offline-only MaxMind lookup; empty if no mmdb)
|
||||
|
||||
#### Caching
|
||||
|
||||
All lookup providers cache results in a local sqlite+pickle DB (default `~/.visidata_cache.db`).
|
||||
|
||||
#### Lookup Providers + Keys
|
||||
|
||||
Options (set in `config.py` / `visidatarc`):
|
||||
- `options.tke_cache_db_path="~/.visidata_cache.db"`
|
||||
- `options.tke_lookup_cache_ttl=86400`
|
||||
- `options.tke_lookup_error_ttl=300`
|
||||
- `options.tke_lookup_timeout=10`
|
||||
- `options.tke_ipinfo_token="..."` (optional; ipinfo can work without it)
|
||||
- `options.tke_ipapi_key="..."` (optional)
|
||||
- `options.tke_vt_api_key="..."` (required for VT lookups unless using `~/.virustotal_api_key`)
|
||||
- `options.tke_maxmind_mmdb_path="/path/to/GeoLite2-City.mmdb"` (optional)
|
||||
|
||||
Env var equivalents:
|
||||
- `IPINFO_TOKEN`, `IPAPI_KEY`
|
||||
- `VT_API_KEY` or `VIRUSTOTAL_API_KEY` (also supports `~/.virustotal_api_key`)
|
||||
- `MAXMIND_MMDB_PATH` or `GEOIP_MMDB_PATH`
|
||||
|
||||
MaxMind (offline “free” GeoLite2) support:
|
||||
- Place a `GeoLite2-City.mmdb` / `GeoLite2-Country.mmdb` file in `$VD_DIR/`, or set `options.tke_maxmind_mmdb_path`.
|
||||
|
||||
### `plugins/iplib.py`
|
||||
|
||||
Pure-Python library used by `iptype.py` for:
|
||||
- Normalized info classes (`IPInfo`, `ASNInfo`, `VTInfo`, `GeoInfo`)
|
||||
- `JSONNode` wrapper (`.data.<field>`) for safe attribute-style access into raw dict/list JSON
|
||||
- Parsing/normalization helpers for each provider’s response shape
|
||||
|
||||
This file intentionally does **not** import VisiData so it can be validated outside the VisiData runtime.
|
||||
|
||||
## Config: `visidatarc`
|
||||
|
||||
This repo’s `visidatarc` is intended to be installed as VisiData’s `config.py`:
|
||||
- `$VD_DIR/config.py` (VisiData 3.3 default)
|
||||
- and also `~/.visidatarc` as a legacy fallback
|
||||
|
||||
It currently contains:
|
||||
- display/date format options
|
||||
- a sqlite+pickle caching decorator and a set of general-purpose helpers (aggregators, timestamp parsing, “dirty” JSON parsing, etc)
|
||||
|
||||
### Are the `visidatarc` functions superseded?
|
||||
|
||||
Partially:
|
||||
- The **IP-centric lookups and normalized attribute access** are now primarily handled by `plugins/iptype.py` on typed values (e.g. `ipcol.geo.country_code`).
|
||||
- Many other helpers in `visidatarc` (aggregators like `avgdiff`, parsing/time conversion helpers, etc.) are still independent and useful.
|
||||
|
||||
### Keeping old + new side-by-side (without duplicating code)
|
||||
|
||||
Yes. The cleanest pattern in VisiData is:
|
||||
1. Put shared logic into a module under `plugins/` (so it’s on `sys.path` via `$VD_DIR`).
|
||||
2. In `visidatarc`, import and expose thin wrappers (or just import the module and use `module.func(...)` in expressions).
|
||||
|
||||
Concretely:
|
||||
- `plugins/iplib.py` already holds parsing/normalization shared by the IP type.
|
||||
- If you have legacy functions in `visidatarc` that overlap with the new IP lookups, refactor those functions into a shared module (e.g. `plugins/lookups.py`) and have both `visidatarc` and `plugins/iptype.py` call into it.
|
||||
|
||||
This keeps backward-compatible names available while ensuring caching/auth/provider behavior is implemented in one place.
|
||||
|
||||
Reference in New Issue
Block a user