Network usage monitor using Python

Created2020-04-10Updated2020-08-23

Previous

We have built a 4G/LTE router but how do I know its network usage? How much will it cost me?

Goal

We are now going to write a Python script to send Email notificatoin for network usage and set a threshold for it.

I assume you know the basic pattern how to code in Python

Requirements.txt

1
2
3
psutil
python_http_client
sendgrid

You can install them by typing in terminal:

1
$ pip3 install psutil python_http_client sendgrid

You may need to specify pip3 in Raspbian as both Python 2.x and Python 3.x were installed.

Coding Python

1
2
3
$ mkdir network-monitor
$ cd network-monitor
$ nano monitor.py

And then we start coding

Get network usage

1
2
3
4
5
6
7
8
import psutil

NETWORK_INTERFACE = 'ppp0'

netio = psutil.net_io_counters(pernic=True)
net_usage = netio[NETWORK_INTERFACE].bytes_sent + netio[NETWORK_INTERFACE].bytes_recv

print(net_usage, "bytes")

It will show you how many bytes it has transfered for upload and download.

If it shows zero, change NETWORK_INTERFACE to wwan0.

Set threshold

1
2
3
4
5
6
7
8
9
10
11
12
13
import psutil

NETWORK_INTERFACE = 'wwan0'
NETWORK_LIMIT = 5000000 # 5GB in SI standard

while True:
  netio = psutil.net_io_counters(pernic=True)
  net_usage = netio[NETWORK_INTERFACE].bytes_sent + netio[NETWORK_INTERFACE].bytes_recv

  if net_usage > NETWORK_LIMIT:
    print("Meets network limit!")

    print(net_usage, "bytes has been used")

Once it over the NETWORK_LIMIT it will print a lot of lines of “Meets network limit!”. We can add a timer to check the network limit every X second.

Separate the config and code

Try to separate the cnofiguratoin and the Python script so we can reuse this project on other devices.

configparser is really a good helper on this.

Create a file monitor.conf

1
2
3
4
5
6
7
8
9
10
11
12
[Email]
SENDGRID_API_KEY = Your.Keyfrom_SendGrid
from = helper@raspberry.pi
to = your@email.address
subject = From your Raspberry Pi

[Network]
INTERFACE = wwan0
LIMIT = 45000000000

[Misc]
TIME_INTER = 36

Now we can load the configuration in Python

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import time
import psutil
import configparser
config = configparser.ConfigParser()
config.optionxform = str  #reference: http://docs.python.org/library/configparser.html
config.read('monitor.conf')

NETWORK_INTERFACE = config.get('Network', 'INTERFACE')		# Define interface to check
NETWORK_LIMIT = int(config.get('Network', 'LIMIT'))			# Define network limit
SENDGRID_API_KEY = config.get('Email', 'SENDGRID_API_KEY')	# Define the API key for SendGrid
TIME_INTER = config.get('Misc', 'TIME_INTER')				# Define time interval 36 seconds

while True:
  time.sleep(TIME_INTER) # Check every 36 seconds
  netio = psutil.net_io_counters(pernic=True)
  net_usage = netio[NETWORK_INTERFACE].bytes_sent + netio[NETWORK_INTERFACE].bytes_recv

  if net_usage > NETWORK_LIMIT:
    print("Meets network limit!")

    print(net_usage, "bytes has been used")

Prepare for the Email

We use SendGrid to send the Email notification in this tutorial.

You may need to sign up on SendGrid. I choose it because of 100 free email quota every day.

Generate SendGrid API Key

The option is in Setting -> API Keys -> Create API Key Creating an API key

Copy the Key and paste to monitor.conf

1
2
[Email]
SENDGRID_API_KEY = Your.Keyfrom_SendGrid

Write python script

Refer to their official guide SendGrid GitHub repo

1
2
3
4
import time
import psutil
import sendgrid
from sendgrid.helpers.mail import *

Read the monitor.conf

1
2
3
4
5
6
7
8
9
10
import configparser
config = configparser.ConfigParser()
config.optionxform = str  #reference: http://docs.python.org/library/configparser.html
config.read('netio-mon.conf')

NETWORK_INTERFACE = config.get('Network', 'INTERFACE')
NETWORK_LIMIT = int(config.get('Network', 'LIMIT'))
NETWORK_MAX = int(config.get('Network', 'MAX'))
SENDGRID_API_KEY = config.get('Email', 'SENDGRID_API_KEY')
TIME_INTER = config.get('Misc', 'TIME_INTER')

I would like to have some loggings too so I added this line

1
2
3
4
5
6
7
8
import logging

loggingFile = logging.FileHandler('my.log', 'w', 'utf-8')

logging.basicConfig(level=logging.DEBUG,
        format='%(asctime)s %(levelname)s %(message)s',
        datefmt='%Y-%m-%d %H:%M',
        handlers=[loggingFile, ])

Declare two methods for sending Email

1
2
3
4
5
6
7
8
9
10
11
def create_message(sender, to, subject, message_text):
  logging.info("send email::" + message_text)
  from_email = Email(sender)
  to_email = To(to)
  subject = subject
  content = Content("text/plain", message_text)
  return Mail(from_email, to_email, subject, content)


def send_message(service, message):
  return service.client.mail.send.post(request_body=message.get())

Now the mean loop to check the network usage periodically.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# Initialize the SendGrid API
service = sendgrid.SendGridAPIClient(api_key=SENDGRID_API_KEY)

# A flag to make sure only send one notification
have_sent = False

while True:
  time.sleep(TIME_INTER)
  netio = psutil.net_io_counters(pernic=True)
  net_usage = netio[NETWORK_INTERFACE].bytes_sent + netio[NETWORK_INTERFACE].bytes_recv
  if net_usage > NETWORK_LIMIT and not have_sent:
    message = create_message(
      config.get('Email', 'from'),
      config.get('Email', 'to'),
      config.get('Email', 'subject'),
      'The network have used %s bytes' %net_usage)
    send_message(service, message)
    have_sent = True

Improvement

  • Prevent network usage lost, save the usage in the last email with pickle
  • In order to monitoring network usage continuously, set one more NETWORK_MAX for reseting the flag after first email notification
    See my GitHub Repo