125 lines
4.8 KiB
Markdown
125 lines
4.8 KiB
Markdown
# VisiData Config + Plugins
|
||
|
||
This folder contains a VisiData `config.py` (symlinked from `visidatarc`) plus a small set of local plugins under `plugins/`.
|
||
|
||
## Install
|
||
|
||
The installer links (or copies) the config and plugins into VisiData’s per-user directory.
|
||
|
||
```bash
|
||
./install.sh --link # default, symlinks into place
|
||
./install.sh --copy # copies into place
|
||
./install.sh --deps # installs optional Python deps into $VD_DIR/plugins-deps
|
||
```
|
||
|
||
On VisiData 3.3, `$VD_DIR` defaults to:
|
||
- macOS: `~/Library/Preferences/visidata`
|
||
- Linux: `${XDG_CONFIG_HOME:-~/.config}/visidata`
|
||
|
||
## Plugins
|
||
|
||
Plugins are installed into `$VD_DIR/plugins/` and imported via the top-level `plugins` package.
|
||
|
||
### `plugins/hidecol.py`
|
||
|
||
Adds a command to hide columns that are empty or constant across all rows.
|
||
|
||
- Command: `tke-hidecol`
|
||
- Menu: `Column -> Hide -> empty/superfluous columns`
|
||
|
||
### `plugins/iptype.py`
|
||
|
||
Adds a custom IP datatype that supports:
|
||
- IPv4 + IPv6 addresses
|
||
- CIDR networks (e.g. `192.168.7.0/24`)
|
||
- Correct sorting (numeric, by version)
|
||
- Membership test operator: `ip * net` (and `net * ip`)
|
||
- Normalized lookup/enrichment properties, accessible as attributes in expressions
|
||
|
||
#### Type + Command
|
||
|
||
- Type converter: `ip(...)`
|
||
- Type name: `IP`
|
||
- Command: `type-ip` (sets `cursorCol.type=ip`)
|
||
|
||
#### Operations
|
||
|
||
Membership test:
|
||
- `ipcol * "192.168.7.0/24"` -> `True`/`False`
|
||
- `"192.168.7.0/24" * ipcol` -> `True`/`False`
|
||
|
||
#### Attributes (on `IP` typed cells)
|
||
|
||
Lookup objects expose both normalized fields and raw response data:
|
||
|
||
- `ipcol.ipinfo.country`
|
||
- `ipcol.ipinfo.data.<any_json_field>`
|
||
- `ipcol.asn.asn`, `ipcol.asn.name`, `ipcol.asn.country`
|
||
- `ipcol.asn.data.<any_json_field>`
|
||
- `ipcol.vt.verdict` (e.g. `"3/94"`), `ipcol.vt.malicious`, `ipcol.vt.total`, `ipcol.vt.category` (alias: `ipcol.vt.type`)
|
||
- `ipcol.vt.data.<any_json_field>`
|
||
- `ipcol.geo.*` (best-available geo: prefers MaxMind mmdb, else free HTTP providers)
|
||
- `ipcol.maxmind.*` (offline-only MaxMind lookup; empty if no mmdb)
|
||
|
||
#### Caching
|
||
|
||
All lookup providers cache results in a local sqlite+pickle DB (default `~/.visidata_cache.db`).
|
||
|
||
#### Lookup Providers + Keys
|
||
|
||
Options (set in `config.py` / `visidatarc`):
|
||
- `options.tke_cache_db_path="~/.visidata_cache.db"`
|
||
- `options.tke_lookup_cache_ttl=86400`
|
||
- `options.tke_lookup_error_ttl=300`
|
||
- `options.tke_lookup_timeout=10`
|
||
- `options.tke_ipinfo_token="..."` (optional; ipinfo can work without it)
|
||
- `options.tke_ipapi_key="..."` (optional)
|
||
- `options.tke_vt_api_key="..."` (required for VT lookups unless using `~/.virustotal_api_key`)
|
||
- `options.tke_maxmind_mmdb_path="/path/to/GeoLite2-City.mmdb"` (optional)
|
||
|
||
Env var equivalents:
|
||
- `IPINFO_TOKEN`, `IPAPI_KEY`
|
||
- `VT_API_KEY` or `VIRUSTOTAL_API_KEY` (also supports `~/.virustotal_api_key`)
|
||
- `MAXMIND_MMDB_PATH` or `GEOIP_MMDB_PATH`
|
||
|
||
MaxMind (offline “free” GeoLite2) support:
|
||
- Place a `GeoLite2-City.mmdb` / `GeoLite2-Country.mmdb` file in `$VD_DIR/`, or set `options.tke_maxmind_mmdb_path`.
|
||
|
||
### `plugins/iplib.py`
|
||
|
||
Pure-Python library used by `iptype.py` for:
|
||
- Normalized info classes (`IPInfo`, `ASNInfo`, `VTInfo`, `GeoInfo`)
|
||
- `JSONNode` wrapper (`.data.<field>`) for safe attribute-style access into raw dict/list JSON
|
||
- Parsing/normalization helpers for each provider’s response shape
|
||
|
||
This file intentionally does **not** import VisiData so it can be validated outside the VisiData runtime.
|
||
|
||
## Config: `visidatarc`
|
||
|
||
This repo’s `visidatarc` is intended to be installed as VisiData’s `config.py`:
|
||
- `$VD_DIR/config.py` (VisiData 3.3 default)
|
||
- and also `~/.visidatarc` as a legacy fallback
|
||
|
||
It currently contains:
|
||
- display/date format options
|
||
- a sqlite+pickle caching decorator and a set of general-purpose helpers (aggregators, timestamp parsing, “dirty” JSON parsing, etc)
|
||
|
||
### Are the `visidatarc` functions superseded?
|
||
|
||
Partially:
|
||
- The **IP-centric lookups and normalized attribute access** are now primarily handled by `plugins/iptype.py` on typed values (e.g. `ipcol.geo.country_code`).
|
||
- Many other helpers in `visidatarc` (aggregators like `avgdiff`, parsing/time conversion helpers, etc.) are still independent and useful.
|
||
|
||
### Keeping old + new side-by-side (without duplicating code)
|
||
|
||
Yes. The cleanest pattern in VisiData is:
|
||
1. Put shared logic into a module under `plugins/` (so it’s on `sys.path` via `$VD_DIR`).
|
||
2. In `visidatarc`, import and expose thin wrappers (or just import the module and use `module.func(...)` in expressions).
|
||
|
||
Concretely:
|
||
- `plugins/iplib.py` already holds parsing/normalization shared by the IP type.
|
||
- If you have legacy functions in `visidatarc` that overlap with the new IP lookups, refactor those functions into a shared module (e.g. `plugins/lookups.py`) and have both `visidatarc` and `plugins/iptype.py` call into it.
|
||
|
||
This keeps backward-compatible names available while ensuring caching/auth/provider behavior is implemented in one place.
|
||
|