Traffic Control Using Linux Kernel 2.4
					======================================

Changelog:

	* Wed Sept 12 2001
	- Added upstream control section
	* Mon Sept 10 2001
	- Initial roll-out

SYNOPSIS:

	Traffic control and shaping in Linux 2.4 using "tc".

DESCRIPTION:

	This document deals with setting up a Linux 2.4 box as a traffic
	shaping machine using the "tc" command part of the "iproute2"
	package. 
	
	This document does not deal with the theory behind traffic shaping
	in Linux.

REQUIREMENTS:
	
	Requirements for RedHat 7.1:

	Kernel 2.4			( RedHat 7.1 )
	iproute-2.2.4-10	( RedHat 7.1 )

	"tc" is part of the "iproute" package in most Linux distributions.
	In case you are recompiling the kernel, make sure the various QOS
	modules are compiled in.

	Requirements for 6.x:

	Distributions which ship with kernel 2.2 also support "tc". I
	however have not tried setting it up tc on legacy kernels.

INSTALLATION:

	Part of standard install on all(?) distributions. Nothing more to be
	installed.

CONFIGURATION:

	Before we can configure the various traffic control rules, we need
	to sit down and form a QOS policy for your LAN. Things to look into
	while formulating a QOS policy:

		- Bandwidth allocated to critical services (HTTP & SMTP)
		- Bandwidth allocated to other services excluding HTTP and SMTP
		- Upstream bandwidth

	For this HOWTO setup, lets assume that the following QOS policy is
	to be implemented:

	On a 1Mbit Link:

		- 50% is to be allocated for HTTP (500Kbit)
		- 40% is to be allocated for remaining services (400Kbit)
		- 5%  is to be allocated for SMTP (50Kbit)
		- 5%  is to be allocated for upstream traffic (50Kbit)
		------
		 100% of 1Mbit
		------

	Some more details:

		- eth0 is on the external WAN side
		- eth1 is on the internal LAN side.
		- Internal LAN is assigned 192.168.2.0/24

Step1: Attach "Class Based Queuing" (CBQ) to eth1

	# tc qdisc add dev eth1 root handle 1: cbq bandwidth 100Mbit avpkt 1000

	Explanation: The above command adds a parent CBQ queuing discipline (1:) to
	the internal interface "eth1" whose total bandwidth is 100Mbit. The NIC used
	here is a 10/100Mbps one and hence the "bandwidth 100Mbit" parameter. If its
	a 10Mbps one, then use "bandwidth 10Mbit" instead.
	
Step2: Add root class

	# tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 100Mbit \
		rate 1Mbit allot 1514 cell 8 weight 100Kbit prio 8 maxburst 20 \
		avpkt 1000 bounded

	Explanation: We now add a parent CBQ class (1:1) under the root class (1:)
	whose total available bandwidth is 100Mbit (bandwidth 100Mbit) and is
	allocated only 1Mbit (rate 1Mbit). The rate is calculated be dividing the
	available bandwidth by a factor of 10. So in this case, the rate is 100Kbit
	(rate 100Kbit).

Step3: Add interactive classes for handling queues

	We now create 3 cbq classes with the following bandwidth allocation:

	1:100 -> 500Kbit (HTTP exclusive use)
	1:200 -> 400Kbit (Other traffic)
	1:300 -> 50Kbit  (SMTP Traffic)
	2:100 -> 50Kbit  (Reserved for upstream traffic)

	Now that we have the root and parent class defines, we now add the
	"interactive" child classes where the actual breakup of the available
	bandwidth of 1Mbit is done.

	# tc class add dev eth1 parent 1:1 classid 1:100 cbq bandwidth 100Mbit \
		rate 500Kbit allot 1514 cell 8 weight 50Kbit prio 3 maxburst 20 \
		avpkt 1000 split 1:0 bounded

	Explanation: A child class (1:100) is created under the parent (1:1). The
	bandwidth allocated is 500Kbit (rate 500Kbit) with a weight of 50Kbit. The
	weight is calculated by diving the rate by a factor of 10. This class which
	has 500Kbit allocated will handle the 

	# tc class add dev eth1 parent 1:1 classid 1:200 cbq bandwidth 100Mbit \
		rate 400Kbit allot 1514 cell 8 weight 40Kbit prio 5 maxburst 20 \
		avpkt 1000 bounded

	Explanation: A second child class (1:200) is created to handle other ports
	except HTTP and has 400Kbit of 1Mbit available for this purpose.

	# tc class add dev eth1 parent 1:1 classid 1:300 cbq bandwidth 100Mbit \
		rate 50Kbit allot 1514 cell 8 weight 5Kbit prio 7 maxburst 20 \
		avpkt 1000 bounded

	Explanation: A third child class (1:300) with an allocated 50Kbit bandwidth
	will handle SMTP traffic.

Step4: Attach "filters" for each of the classes

	What we have done so far is to create queues and distribute the total
	available bandwidth of 1Mbit among them as per our QOS policy. Next, we
	tell the kernel how to manage our queues by attaching "filters" for each of
	our queues.

	# tc qdisc add dev eth1 parent 1:1   sfq quantum 1514b perturb 15
	# tc qdisc add dev eth1 parent 1:100 sfq quantum 1514b perturb 15
	# tc qdisc add dev eth1 parent 1:200 sfq quantum 1514b perturb 15
	# tc qdisc add dev eth1 parent 1:300 sfq quantum 1514b perturb 15

	Explanation: In this case we install the Stochastic Fairness Queueing
	discipline (sfq), which is not quite fair, but works well up to high
	bandwidths without burning up CPU cycles. There are other queuing
	disciplines available which are better, but need more CPU. The Token Bucket
	Filter is often used. 

Step5: Attach "classifiers" to the queues

	Classifiers are the way by which the kernel decides which queue a packet
	should be placed into. There are various different classifiers, each of
	which can be used for different purposes. We will be using 2 classifiers -
	"fw" and "u32".
	
	- "fw" Bases the decision on how the firewall has marked the packet
	- "u32" Bases the decision on fields within the packet (i.e. source IP
	  address, etc) 

	Classifier to match HTTP and SMTP traffic:

	# tc filter add dev eth1 parent 1:0 protocol ip prio 98 u32 match ip \
		sport 0x0050 0xffff flowid 1:100
	# tc filter add dev eth1 parent 1:0 protocol ip prio 97 u32 match ip \
		sport 0x0019 0xffff flowid 1:300

	Explanation: The "u32" filter catches all HTTP (0x0050 = 80) and SMTP
	traffic (0x0019 = 25) and directs them to the appropriate queues:

	- Port 80 (0x0050) -> 1:100 -> 500Kbit queue
	- Port 25 (0x0019) -> 1:300 -> 50Kbit queue

	Classifier to catch all other traffic:

	A generic filter for our internal LAN (192.168.2.0/24) is added with a
	higher priority so that all unmatched traffic for 1:100 and 1:300 will be
	directed to the 400Kbit queue (1:200). Priority matters, lower priority =
	higher preference.

	# tc filter add dev eth1 parent 1:0 protocol ip prio 99 u32 match ip \
	dst 192.168.2.0/24 flowid 1:200

	Explanation: Catch all traffic for destination IP of 192.168.2.0/24 and
	direct to the 400Mbit queue (1:200). The priority assigned (prio 99) is a
	very high one so that other filters can be matched before the generic one is
	applied.
	
	To maintain flexibility in the shaping rules, we also add some firewall "fw"
	filters so that we can use iptables/ipchains in the matching process. Packet
	marking works like this:

	- Mark packets using ipchains/iptables using rules
	- Use "fw" filter to match marked packets
	- Direct marked packets to appropriate queues

	# tc filter add dev eth1 protocol ip parent 1:0 prio 10 handle 1 fw \
	classid 1:100
	# tc filter add dev eth1 protocol ip parent 1:0 prio 11 handle 2 fw \
	classid 1:200
	# tc filter add dev eth1 protocol ip parent 1:0 prio 12 handle 3 fw \
	classid 1:300

	Explanation: All packets with mark 1, 2 or 3 will be forwarded to queues
	1:100, 1:200 or 1:300 respectively.

Marking Packets:
===============

	Here is a sample iptables rule for marking packets:

	# iptables -I PREROUTING -t mangle -p tcp -d HostA -j MARK --set-mark 1

	Explanation: Here all TCP packets from HostA will be marked "1" and will be
	handles by the 500Kbit queue (1:100)
	
	Shaping Upstream Traffic
	======================

	In our QOS policy, we have allocated 5% of the effective bandwidth (50Kbps)
	for upstream. For shaping upstream traffic, we need to create a new set of
	queues, classes and filters on the external interface eth0.

	For brevity's sake, all in one go:

	# tc qdisc add dev eth0 root handle 2: cbq bandwidth 100Mbit avpkt 1000
	# tc class add dev eth0 parent 2:0 classid 2:1 cbq bandwidth 100Mbit \
		rate 1Mbit allot 1514 cell 8 weight 100Kbit prio 8 maxburst 20 \
		avpkt 1000 bounded
	# tc class add dev eth0 parent 2:1 classid 2:100 cbq bandwidth 100Mbit \
		rate 50Kbit allot 1514 cell 8 weight 5Kbit prio 7 maxburst 20 \
		avpkt 1000 bounded
	# tc qdisc add dev eth0 parent 2:1   sfq quantum 1514b perturb 15
	# tc qdisc add dev eth0 parent 2:100 sfq quantum 1514b perturb 15
	# tc filter add dev eth0 parent 2:0 protocol ip prio 100 u32 match ip \
		src 192.168.2.0/24 flowid 2:100

	EXPLANATION: We create a new root (2: of 1Mbit bandwidth), a new parent (2:1
	of 100Kbit bandwidth) and a new child (2:100 of 50Kbit bandwidth). The
	filter "u32" will match the source network address of 192.168.2.0/24 and
	shape it to 50Kbit by using the 2:100 50Kbit queue.

	The parent queue (2:1) has been allocated a maximum of 100Kbit, if at any
	point of time we need to allocate more upstream bandwidth, we can easily do
	so since the parent class can provide 50Kbit of extra b/w.
	
TESTING:

	Use iptraf or other traffic monitoring tools for a breakup.

FILES:

	For your convenience, i am providing a sysV style init script with the
	rules laid in.

	cbq.rhl (For RedHat users)

SEE ALSO:

	1. Linux 2.4 Advanced Routing HOWTO
	http://www.linuxdoc.org/HOWTO/Adv-Routing-HOWTO.html

	2. TC command reference
	http://linux.bergqvist.se/

TODO:

	currently we are shaping only incoming traffic. We need to add rules on the
	external interface (eth0) for shaping outgoing traffic.