IpSurv Overview

Installation

$ pip install ipsurv
or
$ pip3 install ipsurv

Features

  • Grouping by IP or Subnet.

  • Skip duplicate by the group.

  • Autodetect IP in line. Trying to resolve the name automatically.

  • Autodetect delimiter-char.

  • Customize output format. There are many format.

  • Output JSON format.

  • Show headers.

  • Check ICMP, TCP, UDP, HTTP.

  • Set timeout.

  • Load env variable. And changing arguments and internal configures.

  • Use GeoIP2 via IpSurv optionally.

Usage

Specify IP using Argument

$ cat ips.txt|ipsurv 192.168.1.10
$ cat ips.txt|ipsurv 192.168.1.10 192.168.1.11

Specify IP using PIPE

$ cat ips.txt|ipsurv
$ cat apache.log|ipsurv

Self mode

$ ipsurv self

Example options

$ cat ips.txt|ipsurv --group=24
$ cat ips.txt|ipsurv --group=network
$ cat ips.txt|ipsurv --format=country
$ cat ips.txt|ipsurv --format=hostname
$ cat ips.txt|ipsurv --format="{country},{name},{geo}"
$ cat ips.txt|ipsurv --format="{country},{ip_int},{handle},{port43}"
$ cat /var/log/httpd/access_log|ipsurv --ident --no_original

$ cat ips.txt|ipsurv --group=255.255.255.0
$ cat ips.txt|ipsurv --delimiter="\t"
$ cat ips.txt|ipsurv --format="{group}\t{ip_int}\t{country}\t{handle}\t{port43}"
$ cat ips.txt|ipsurv --format="{country},{ip_int},{handle},{port43},{icmp},{port},{tcp}" --group=network --icmp=1 --tcp=1 --timeout=2

Command examples

$ cat .dev/test/government.txt|ipsurv --headers=1 --format="{status},{group},{country},{name},{cidr},{http},{http_h2}" --group=network --http=2

original,status,group,country,name,cidr,http,http_h2
www.whitehouse.gov,OK,192.0.64.1,US,AUTOMATTIC,192.0.64.0/18,HTTP_OK,HTTP2
www.state.gov,OK,3.165.0.1,US,AMAZON-CF,3.165.0.0/16,HTTP_OK,HTTP2
www.treasury.gov,OK,23.32.0.1,US,AKAMAI,23.32.0.0/11,HTTP_OK,HTTP2
www.gov.uk,OK,151.101.0.1,US,SKYCA-3,151.101.0.0/16,HTTP_OK,HTTP2
www.gouvernement.fr,OK,217.70.184.1,FR,GANDIFR-NET4,217.70.184.0/24,HTTP_OK,HTTP1
www.diplomatie.gouv.fr,OK,77.128.0.1,FR,FR-SFR-20100831,77.128.0.0/11,HTTP_OK,HTTP1
www.economie.gouv.fr,OK,141.101.88.1,EU,CLOUDFLARE-EU,141.101.88.0/21,HTTP_OK,HTTP2
www.bundesregierung.de,OK,185.173.230.1,DE,BABIEL-NET-230,185.173.230.0/24,HTTP_OK,HTTP2

For details, Command arguments / Command examples

Using GeoIP2 optionally

IpSurv support GeoIP2 optionally. If GeoIP2 module is installed and there are mmdb files, You can use GeoIP2 features via IpSurv. When using the GeoIP2 module, network communication does not occur, which enables faster processing.

Please refer to the Using GeoIP2 documentation in detail.

Examples

$ ipsurv 8.8.8.8 --geoip_only
8.8.8.8,NA,North America,US,United States,America/Chicago,AS15169,37.751;-97.822

$ ipsurv www.wikipedia.org --format="{country},{geo}" --collect=geoip --headers=1
original,country,geo
www.wikipedia.org,US,37.751;-97.822

$ ipsurv 8.8.8.8 --format="{continent},{continent_name},{country},{geo}" --collect="geoip" --headers=1
original,continent,continent_name,country,geo
8.8.8.8,NA,North America,US,37.751;-97.822

> IpSuv support customizing GeoIP2 data path by IPSURV_CONF env. Please read here about IPSURV_CONF.

Debugging

In verbose mode, outputting internal data and behaviors in detail.

$ ipsurv ***** --verbose=2     #INFO
$ ipsurv ***** --verbose=3     #DEBUG

$ ipsurv ***** --debug      #DEBUG  This option is equivalent to "--verbose=3"

For details, Development and Debugging

Customizing IpSurv

`IpSurv` is implemented as customizable program architecture. `IpSurv` provide extending features and several classes. Please read `program_architecture_classes.md`.

For details, Program architecture and Classes / Development and Debugging / Customizing and Examples