gists/config/visidata

VisiData Config + Plugins

This folder contains a VisiData config.py (symlinked from visidatarc) plus a small set of local plugins under plugins/.

Install

The installer links (or copies) the config and plugins into VisiData’s per-user directory.

./install.sh --link        # default, symlinks into place
./install.sh --copy        # copies into place
./install.sh --deps        # installs optional Python deps into $VD_DIR/plugins-deps

On VisiData 3.3, $VD_DIR defaults to:

• macOS: ~/Library/Preferences/visidata
• Linux: ${XDG_CONFIG_HOME:-~/.config}/visidata

Plugins

Plugins are installed into $VD_DIR/plugins/ and imported via the top-level plugins package.

Showcase Demo

This repo includes a self-contained sample dataset + command log to demonstrate the local IOC/IP features:

• showcase_ioc.tsv (sample IOC rows)
• showcase_ioc.vdj (replay file)

Run it interactively from this repo root:

vd --visidata-dir "$PWD" --config "$PWD/visidatarc" --play showcase_ioc.vdj

What it showcases:

• custom types: IP, Domain, URL, Hash
• IP membership expressions: src_ip * network
• IP network fields: src_ip.type, src_ip.mask, src_ip.range, src_ip.broadcast, src_ip.identity, src_ip.hostcount, src_ip.rfc_type
• URL parsing fields: url.host, url.parts.path, url.domain
• hash classification: file_hash.kind
• IP lookups: src_ip.ipinfo.*, src_ip.asn.*, src_ip.geo.*, src_ip.country()
• provider visibility: src_ip.geo.source, src_ip.asn.source, domain.dns.source
• domain/network intel: domain.dns.*, domain.rdap.*, domain.resolveip, domain.resolveips, domain.resolveipv4, domain.resolveipv6
• hash intel: file_hash.mb.* (MalwareBazaar)
• VirusTotal lookups: src_ip.vt.*, file_hash.vt.*, domain.vt.*, url.vt.* (plus hash.vt.name, hash.vt.names, hash.vt.score, domain.vt.ip, domain.vt.ips)
• local plugin command: tke-hidecol

Lookup notes:

• VT columns require options.tke_vt_api_key (or VT_API_KEY / VIRUSTOTAL_API_KEY / ~/.virustotal_api_key).
• IPInfo/ASN/Geo columns use free providers and may be rate-limited; options.tke_ipinfo_token improves reliability.
• To keep replays practical with strict throttling, some heavy lookup columns are intentionally limited to a subset of rows.

plugins/hidecol.py

Adds a command to hide columns that are empty or constant across all rows.

• Command: tke-hidecol
• Menu: Column -> Hide -> empty/superfluous columns

plugins/iptype.py

Adds a custom IP datatype that supports:

• IPv4 + IPv6 addresses
• CIDR networks (e.g. 192.168.7.0/24)
• Correct sorting (numeric, by version)
• Membership test operator: ip * net (and net * ip)
• Normalized lookup/enrichment properties, accessible as attributes in expressions

Type + Command

• Type converter: ip(...)
• Type name: IP
• Command: type-ip (sets cursorCol.type=ip)

Operations

Membership test:

• ipcol * "192.168.7.0/24" -> True/False
• "192.168.7.0/24" * ipcol -> True/False

Attributes (on IP typed cells)

Lookup objects expose both normalized fields and raw response data:

• ipcol.type (ipv4/ipv6/cidr4/cidr6), ipcol.family, ipcol.is_cidr
• ipcol.mask, ipcol.netmask, ipcol.identity, ipcol.broadcast, ipcol.range, ipcol.hostcount, ipcol.address_count
• ipcol.rfc_type (classification: e.g. global, private, documentation, shared, link-local, ...)
• ipcol.ipinfo.country
• ipcol.ipinfo.data.<any_json_field>
• ipcol.asn.asn, ipcol.asn.name, ipcol.asn.country
• ipcol.asn.data.<any_json_field>
• ipcol.vt.verdict (e.g. "3/94"), ipcol.vt.score, ipcol.vt.malicious, ipcol.vt.total, ipcol.vt.category (alias: ipcol.vt.type)
• ipcol.vt.data.<any_json_field>
• ipcol.geo.* (best-available geo: prefers MaxMind mmdb, else free HTTP providers)
• ipcol.maxmind.* (offline-only MaxMind lookup; empty if no mmdb)

Type shortcuts on table-like sheets:

• ;i -> type-ip
• ;d -> type-domain
• ;u -> type-url-ioc
• ;h -> type-hash

Caching

All lookup providers cache results in a local sqlite+pickle DB (default ~/.visidata_cache.db).

Lookup Providers + Keys

Options (set in config.py / visidatarc):

• options.tke_cache_db_path="~/.visidata_cache.db"
• options.tke_lookup_cache_ttl=86400
• options.tke_lookup_error_ttl=300
• options.tke_lookup_timeout=10
• options.tke_ipinfo_token="..." (optional; ipinfo can work without it)
• options.tke_ipapi_key="..." (optional)
• options.tke_vt_api_key="..." (required for VT lookups unless using ~/.virustotal_api_key)
• options.tke_maxmind_mmdb_path="/path/to/GeoLite2-City.mmdb" (optional)

Env var equivalents:

• IPINFO_TOKEN, IPAPI_KEY
• VT_API_KEY or VIRUSTOTAL_API_KEY (also supports ~/.virustotal_api_key)
• MAXMIND_MMDB_PATH or GEOIP_MMDB_PATH

MaxMind (offline “free” GeoLite2) support:

• Place a GeoLite2-City.mmdb / GeoLite2-Country.mmdb file in $VD_DIR/, or set options.tke_maxmind_mmdb_path.

plugins/iplib.py

Pure-Python library used by iptype.py for:

• Normalized info classes (IPInfo, ASNInfo, VTInfo, GeoInfo)
• JSONNode wrapper (.data.<field>) for safe attribute-style access into raw dict/list JSON
• Parsing/normalization helpers for each provider’s response shape

This file intentionally does not import VisiData so it can be validated outside the VisiData runtime.

VT schema (*.vt)

ip.vt, domain.vt, url.vt, and hash.vt expose a normalized shape for quick querying across free + premium responses, while still preserving full raw JSON:

Common fields:

• verdict ("malicious/total")
• score / confidence (malicious/total float)
• malicious, suspicious, harmless, undetected, timeout, total
• category / categories
• reputation, votes_harmless, votes_malicious
• tags, last_analysis_date, last_modification_date
• results (normalized engine results map), stats, data (full raw API response)

Object-specific conveniences:

• ip.vt: asn, as_owner, country, continent, network
• domain.vt: ip (best/last known), ips (all extracted A/AAAA)
• url.vt: URL-level verdict/score plus direct raw access via url.vt.attrs.*
• hash.vt: name (best malware name), names (all extracted names), plus verdict/score

Raw passthrough:

• Any VT attributes field is also available via obj.vt.<attribute_name> and obj.vt.attrs.<attribute_name>.

Config: visidatarc

This repo’s visidatarc is intended to be installed as VisiData’s config.py:

• $VD_DIR/config.py (VisiData 3.3 default)
• and also ~/.visidatarc as a legacy fallback

It currently contains:

• display/date format options
• a sqlite+pickle caching decorator and a set of general-purpose helpers (aggregators, timestamp parsing, “dirty” JSON parsing, etc)

Are the visidatarc functions superseded?

Partially:

• The IP-centric lookups and normalized attribute access are now primarily handled by plugins/iptype.py on typed values (e.g. ipcol.geo.country_code).
• Many other helpers in visidatarc (aggregators like avgdiff, parsing/time conversion helpers, etc.) are still independent and useful.

Keeping old + new side-by-side (without duplicating code)

Yes. The cleanest pattern in VisiData is:

1. Put shared logic into a module under plugins/ (so it’s on sys.path via $VD_DIR).
2. In visidatarc, import and expose thin wrappers (or just import the module and use module.func(...) in expressions).

Concretely:

• plugins/iplib.py already holds parsing/normalization shared by the IP type.
• If you have legacy functions in visidatarc that overlap with the new IP lookups, refactor those functions into a shared module (e.g. plugins/lookups.py) and have both visidatarc and plugins/iptype.py call into it.

This keeps backward-compatible names available while ensuring caching/auth/provider behavior is implemented in one place.