7390 + Downstream QOS

[terpstra]

Good evening!

I recently bought a Fritz!Box 7390 to replace my aging Asus WL-500g (with openwrt). My DSL is very slow; 200kB/s downstream and 20kB/s upstream. Whenever downloading something, the massive buffers at the ISP fill up and I see upwards of 3s of latency to ping google. With the WL-500g I had made a custom QOS script which did an excellent job of traffic shaping, bringing this down to a manageable 80ms at the cost of some 20kB/s of download speed. I achieved this by using the IMQ kernel patch to drop incoming TCP packets over my 180kB/s budget to throttle download speeds and keep the ISP's buffers clear. I want to do something similar on the new Fritz!Box. Unfortunately, the official firmware totally failed in this regard, so now I've installed Freetz.

Unfortunately, Freetz on the 7390 does not seem to have a PPP network interface and/or the traffic somehow is missing?? I can't find an interface where tcpdump shows packets during a download. Even if I inspect the "lan" interface, all the TCP data packets supposedly have length=0 and don't appear with the rate I would expect. Is there some sort of TCP offload hardware / am I going to even be able to achieve reasonable downstream traffic shaping? Setting up the script and compiling the kernel is not a problem. My problem is that it seems linux doesn't even SEE the download traffic.

It's not too late for me to return this router. If I can't get downstream QOS working, I'd like to know as soon as possible, as the (quite overpriced) unit is then useless to me.

 

[terpstra]

So, I got it to work, finally.

There are two problems:
1- The device 'adsl' is atm and doesn't play well with any normal tools
2- There is an AVM packet accelerator that hides packets from linux

The solution to get working downstream qos:
1- Disable avm_pa: echo disable > /proc/net/avm_pa/control
(thank god the source code for avm_pa was available!)
2- Compile ifb.ko and act_mirred.ko
3- Recompile a full featured tc and install as tc2
3- Redirect external traffic on cpmac0 (ethernet) and ath0 (wireless) to ifb0

I did this using iptabes, but you could also use "u32" instead of "fw":
tc2 qdisc del dev cpmac0 root 2> /dev/null
tc2 qdisc add dev cpmac0 handle 1:0 root hfsc
tc2 filter add dev cpmac0 parent 1:0 protocol ip prio 6 handle 100 fw flowid 1:1 action mirred egress redirect dev ifb0

tc2 qdisc del dev ath0 root 2> /dev/null
tc2 qdisc add dev ath0 handle 1:0 root hfsc
tc2 filter add dev ath0 parent 1:0 protocol ip prio 6 handle 100 fw flowid 1:1 action mirred egress redirect dev ifb0

iptables -t mangle -A POSTROUTING ! -s 192.168.0.0/16 -j MARK --set-mark 100 # go to ifb0

4- Enable QoS on the combined egress interface
ifconfig ifb0 up
tc qdisc del dev ifb0 root
tc qdisc add dev ifb0 root handle 1: tbf rate 1600kbit burst 8192b lat 10ms
... substitute your bandwidth in the 'rate' field.

 

[terpstra]

Here is a patch that makes the necessary changes to freetz-trunk:

Index: make/linux/Config.iks.7390_05.50
===================================================================
--- make/linux/Config.iks.7390_05.50	(revision 10304)
+++ make/linux/Config.iks.7390_05.50	(working copy)
@@ -1,7 +1,7 @@
 #
 # Automatically generated make config: don't edit
 # Linux kernel version: 2.6.28.8
-# Thu Mar 29 21:24:39 2012
+# Tue Apr  2 22:36:49 2013
 #
 CONFIG_MIPS=y
 
@@ -501,6 +501,7 @@
 # CONFIG_NET_SCH_DSMARK is not set
 # CONFIG_NET_SCH_NETEM is not set
 CONFIG_NET_SCH_LLQ=m
+CONFIG_NET_SCH_INGRESS=m
 
 #
 # Classification
@@ -510,15 +511,24 @@
 CONFIG_NET_CLS_TCINDEX=m
 CONFIG_NET_CLS_ROUTE4=m
 CONFIG_NET_CLS_ROUTE=y
-# CONFIG_NET_CLS_FW is not set
+CONFIG_NET_CLS_FW=m
 CONFIG_NET_CLS_U32=m
 # CONFIG_CLS_U32_PERF is not set
-# CONFIG_CLS_U32_MARK is not set
+CONFIG_CLS_U32_MARK=y
 # CONFIG_NET_CLS_RSVP is not set
 # CONFIG_NET_CLS_RSVP6 is not set
 # CONFIG_NET_CLS_FLOW is not set
 # CONFIG_NET_EMATCH is not set
-# CONFIG_NET_CLS_ACT is not set
+CONFIG_NET_CLS_ACT=y
+CONFIG_NET_ACT_POLICE=m
+CONFIG_NET_ACT_GACT=m
+# CONFIG_GACT_PROB is not set
+CONFIG_NET_ACT_MIRRED=m
+# CONFIG_NET_ACT_IPT is not set
+# CONFIG_NET_ACT_NAT is not set
+# CONFIG_NET_ACT_PEDIT is not set
+# CONFIG_NET_ACT_SIMP is not set
+# CONFIG_NET_ACT_SKBEDIT is not set
 # CONFIG_NET_CLS_IND is not set
 CONFIG_NET_SCH_FIFO=y
 
@@ -770,6 +780,7 @@
 # CONFIG_IEEE1394 is not set
 # CONFIG_I2O is not set
 CONFIG_NETDEVICES=y
+CONFIG_IFB=m
 CONFIG_DUMMY=m
 # CONFIG_BONDING is not set
 # CONFIG_MACVLAN is not set
Index: make/Config.in
===================================================================
--- make/Config.in	(revision 10304)
+++ make/Config.in	(working copy)
@@ -43,6 +43,7 @@
 source make/fuse/Config.in
 source make/haserl/Config.in
 source make/inetd/Config.in
+source make/iproute2/Config.in
 source make/knock/Config.in
 source make/mc/Config.in
 source make/mtr/Config.in
Index: make/iproute2/Makefile.in
===================================================================
--- make/iproute2/Makefile.in	(revision 0)
+++ make/iproute2/Makefile.in	(revision 0)
@@ -0,0 +1,3 @@
+ifeq ($(strip $(FREETZ_PACKAGE_IPROUTE2)),y)
+PACKAGES+=iproute2
+endif
Index: make/iproute2/patches/001-Makefile.patch
===================================================================
--- make/iproute2/patches/001-Makefile.patch	(revision 0)
+++ make/iproute2/patches/001-Makefile.patch	(revision 0)
@@ -0,0 +1,20 @@
+--- Makefile.orig      2005-07-09 00:08:47.000000000 +0200
++++ Makefile   2010-06-20 00:40:30.000000000 +0200
+@@ -10,7 +10,7 @@
+ DEFINES= -DRESOLVE_HOSTNAMES
+ 
+ #options if you have a bind>=4.9.4 libresolv (or, maybe, glibc)
+-LDLIBS=-lresolv
++#LDLIBS=-lresolv
+ ADDLIB=
+ 
+ #options for decnet
+@@ -27,7 +27,7 @@
+ 
+ LDLIBS += -L../lib -lnetlink -lutil
+ 
+-SUBDIRS=lib ip tc misc netem
++SUBDIRS=lib ip tc
+ 
+ LIBNETLINK=../lib/libnetlink.a ../lib/libutil.a
+
Index: make/iproute2/Config.in
===================================================================
--- make/iproute2/Config.in	(revision 0)
+++ make/iproute2/Config.in	(revision 0)
@@ -0,0 +1,5 @@
+config FREETZ_PACKAGE_IPROUTE2
+	bool "iproute2"
+	default n
+	help
+		iproute2 is the advanced ip and tc tools   
Index: make/iproute2/iproute2.mk
===================================================================
--- make/iproute2/iproute2.mk	(revision 0)
+++ make/iproute2/iproute2.mk	(revision 0)
@@ -0,0 +1,39 @@
+$(call PKG_INIT_BIN, 2.6.16-060323)
+$(PKG)_SOURCE:=$(pkg)-$($(PKG)_VERSION).tar.gz
+$(PKG)_SITE:=http://www.jzab.de/files/iproute2/
+#$(PKG)_SITE:=http://devresources.linux-foundation.org/dev/iproute2/download/ - currently down :(
+
+$(PKG)_TC_BINARY:=$($(PKG)_DIR)/tc/tc
+$(PKG)_TC_TARGET_BINARY:=$($(PKG)_DEST_DIR)/sbin/tc2
+$(PKG)_IP_BINARY:=$($(PKG)_DIR)/ip/ip
+$(PKG)_IP_TARGET_BINARY:=$($(PKG)_DEST_DIR)/bin/ip
+
+$(PKG_SOURCE_DOWNLOAD)
+$(PKG_UNPACKED)
+$(PKG_CONFIGURED_NOP)
+
+$($(PKG)_TC_BINARY) $($(PKG)_IP_BINARY): $($(PKG)_DIR)/.configured
+	PATH="$(TARGET_PATH)" \
+	$(MAKE) CC="$(TARGET_CC)" \
+		CROSS_COMPILE="$(TARGET_MAKE_PATH)/$(TARGET_CROSS)" \
+		EXTRA_CFLAGS="$(TARGET_CFLAGS)" \
+		ARCH="mipsel" \
+		-C $(IPROUTE2_DIR)
+
+$($(PKG)_TC_TARGET_BINARY): $($(PKG)_TC_BINARY)
+	$(INSTALL_BINARY_STRIP)
+
+$($(PKG)_IP_TARGET_BINARY): $($(PKG)_IP_BINARY)
+	$(INSTALL_BINARY_STRIP)
+
+$(pkg):
+
+$(pkg)-precompiled: $($(PKG)_TC_TARGET_BINARY) $($(PKG)_IP_TARGET_BINARY)
+
+$(pkg)-clean:
+	-$(MAKE) -C $(IPROUTE2_DIR) clean
+
+$(pkg)-uninstall:
+	$(RM) $(IPROUTE2_TC_TARGET_BINARY) $(IPROUTE2_IP_TARGET_BINARY)
+
+$(PKG_FINISH)
\ No newline at end of file
Index: kernel/Config.in.modules
===================================================================
--- kernel/Config.in.modules	(revision 10304)
+++ kernel/Config.in.modules	(working copy)
@@ -524,6 +524,18 @@
 	select FREETZ_MODULE_l2cap
 	default n
 
+config FREETZ_MODULE_act_police
+	bool "act_police.ko"
+	default n
+	help
+		act_police allows tc qdisc QoS to limit traffic.
+
+config FREETZ_MODULE_act_mirred
+	bool "act_mirred.ko"
+	default n
+	help
+		Make it possible for traffic shaping to use redirect packets.
+
 config FREETZ_MODULE_cls_u32
 	bool "cls_u32.ko"
 	default n
@@ -532,6 +544,19 @@
 		packet, so it is in some ways the most powerful filter provided
 		by the Linux traffic control engine.
 
+config FREETZ_MODULE_cls_fw
+	bool "cls_fw.ko"
+	default n
+	help
+		The fw filter allows you to match on based on the firewall mark
+		created by iptables.
+
+config FREETZ_MODULE_ifb
+	bool "ifb.ko"
+	default n
+	help
+		Makes it possible to use iptables on ingress queues.
+
 config FREETZ_MODULE_ipv6
 	bool "ipv6.ko"
 	depends on FREETZ_TARGET_IPV6_SUPPORT && ! FREETZ_HAS_AVM_IPV6
@@ -574,6 +599,12 @@
 		HTB is very similar to the CBQ regarding its goals however it has
 		different properties and different algorithm.
 
+config FREETZ_MODULE_sch_hfsc
+	bool "sch_hfsc.ko"
+	default n
+	help
+		HFSC
+
 config FREETZ_MODULE_sch_llq
 	bool "sch_llq.ko"
 	default n
@@ -602,6 +633,13 @@
 		See the top of <file:net/sched/sch_tbf.c> for a description of the
 		TBF algorithm.
 
+config FREETZ_MODULE_sch_ingress
+	bool "sch_ingess.ko"
+	default n
+	help
+		Allow traffic shaping on the ingress queue of network interfaces.
+		This makes it possible to drop packets to trigger TCP congestion control.
+
 endmenu
 
 config FREETZ_MODULE_rfcomm

 

[terpstra]

So happy, now. Got two downloads going and ping to google is 60ms average, 120ms max.
Finally the relatives won't ruin my guild wars 2 experience!

For reference, without downstream QoS, I had average 2.5s ping spiking to 4s. So this is around a 40* improvement. Makes web pages load nice and snappy again as well.

 

[ich.hasse.o2]

Thank you for this excellent post. This is exactly what I am looking for. I tried to do it myself, but it was too hard. I tried to set up a qdisc with two classes an two filters to classify the traffic, but i failed. I have some questions:

1. When try too use filters like u32 or fw, I get the error 'Unknown filter "u32", hence option "match" is unparsable'. Lsmod tells me cls_u32.ko is loaded. I do not understand it.
2. You say the tc command of the fritzbox 7390 lacks features. Could that be the reason I cannot use filters?
3. Why do I need the kernel modules ifb and act_mirrored?
4. Is there any way a normal user can understand this or do you need to be a software developer to get this done? I really try hard, but could not understand the source code of the avm_pa.
5. Is there an easy way to cross compile and install the missing features?

Any help would be greatly appreciated, as this is a really common problem. The whole web is full of people who want to control the downstreams of their home network. All needed pieces of the puzzle are there, but i cant get them together. I feel I am really stuck here.

P.S. Antworten auf deutsch wären genauso hilfreich.

 

[terpstra]

1. When try too use filters like u32 or fw, I get the error 'Unknown filter "u32", hence option "match" is unparsable'. Lsmod tells me cls_u32.ko is loaded. I do not understand it.
2. You say the tc command of the fritzbox 7390 lacks features. Could that be the reason I cannot use filters?

Yes. You need to recompile 'tc'. The one in the 7390 does not support the features you need, hence problem #1.

3. Why do I need the kernel modules ifb and act_mirrored?

You cannot shape the DSL link because it is somehow strange. At least I didn't succeed.
My solution instead shapes the ethernet and wireless traffic that will go in/out of the DSL link. To do that you need to be able to combine the traffic to one interface. The ifb+act_mirrored modules make this possible. ifb0 is the virtual interface which we setup to carry the same traffic that the DSL link carries. The act_mirror module is used to copy the traffic to ifb0.

4. Is there any way a normal user can understand this or do you need to be a software developer to get this done? I really try hard, but could not understand the source code of the avm_pa.
5. Is there an easy way to cross compile and install the missing features?

You can build the firmware using the 'freetz' project. That's how I did it. It definitely helps if you are a software developer.

Any help would be greatly appreciated, as this is a really common problem. The whole web is full of people who want to control the downstreams of their home network. All needed pieces of the puzzle are there, but i cant get them together. I feel I am really stuck here.

I feel your pain. What I don't understand is why the AVM firmware doesn't do this itself. They are in a position to shape the DSL traffic directly as they have the source to their custom dsld. It would be much easier for them to achieve downstream QoS than it was for me. It would also be a huge competitive advantage since practically every router (except the open source ones) suck in this regard. I would then recommend the Fritz box to everyone.

PS: I hate O2 too.

 

[ich.hasse.o2]

Thank you for your quick reply!

As I am not using wireless, can I just shape the traffic of cpmac0? As far as I understand now, I only need to disable the avm_pa and to install a working tc. But how do I get a working tc? Do I need to cross compile iproute2 or is there an easy way, such as an option when building freetz?

What I don't understand is why the AVM firmware doesn't do this itself. They are in a position to shape the DSL traffic directly as they have the source to their custom dsld. It would be much easier for them to achieve downstream QoS than it was for me. It would also be a huge competitive advantage since practically every router (except the open source ones) suck in this regard. I would then recommend the Fritz box to everyone.

It is really strange. It would be such a small step to enable this and there are so many requests for it.

 

[terpstra]

As I am not using wireless, can I just shape the traffic of cpmac0? is there an easy way, such as an option when building freetz?

When I say wireless I mean like using a laptop without a cable, not UMTS. You sure you don't need wireless? If you apply my patch the freetz trunk, you should be able to get a working tc in your firmware.

 

[ich.hasse.o2]

I got your patch applied, compiled and flashed. All the needed kernel modules are loaded, but I still have no clue how to get a working tc command. Could you post your binary or do I need more than just the binary? About the wireless question: There is absolutely no wireless involved.

 

[terpstra]

in a rush atm:

make menuconfig
packages
standard packages
iproute2

My patches install it as 'tc2' (you need to normal 'tc' for the AVM stuff).

EDIT: I just realized you will still need ifb+sch_mirrored. sch_egress is shit. Using ifb you can use ingress shaping on egress like I do in my example scripts above.

 

[ich.hasse.o2]

Hi terpstra, first of all thank you for your patience. With your help i managed to get tc and the kernel modules to work. However, i still could not set up the traffic shaping. I thought understood, but it did not work. Here is what I did:

1. Disabled the avm_pa by doing echo disable > /proc/net/avm_pa/control
2. Loaded the kernel modules

I needed to use "replace kernel" to get them to load. With everything prepared, i tried to set up the shaping.

3. Create the root qdisc
tc2 qdisc add dev cpmac0 root handle 1:0 htb

4. Create a class (my dsl downlink has 7 mbit)
tc2 class add dev cpmac0 parent 1: classid 1:1 htb rate 7mbit ceil 7mbit

5. Create a subclass which is throttled to 3 mbit with the name 1:10
tc2 class add dev cpmac0 parent 1:1 classid 1:10 htb rate 3mbit ceil 7mbit

6. Now classify my packages (I am 192.168.0.20) as 1:10
tc2 filter add dev cpmac0 parent 1: protocol ip u32 match ip dst 192.168.0.20 flowid 1:10

But there is no effect, i can still download at 7 mbit. Something is wrong in there. There are no error messages at all, it simply has not the desired effect.

edit: I tried you example but could not figure out how to get an ifb network device

 

[terpstra]

tc2 qdisc add dev cpmac0 root handle 1:0 htb
tc2 class add dev cpmac0 parent 1: classid 1:1 htb rate 7mbit ceil 7mbit

edit: I tried you example but could not figure out how to get an ifb network device

Hmm. I'm not sure what went wrong with your script. I've attached mine below.
You should get an ifb0 as soon as you load the ifb.ko module. It won't be 'up'.
Try ifconfig ifb0 before and after loading ifb.ko.

One thing I noticed is that the AVM kernel has very limited memory for modules. If you load too many you start getting error messages (check dmesg). I load exactly the modules I needed and no others. Try rebooting your router and then running these commands:

#! /bin/bash

modprobe ifb
echo disable > /proc/net/avm_pa/control

tc2 qdisc del dev cpmac0 root 2> /dev/null
tc2 qdisc add dev cpmac0 handle 1:0 root hfsc
tc2 filter add dev cpmac0 parent 1:0 protocol ip prio 6 handle 100 fw flowid 1:1 action mirred egress redirect dev ifb0

tc2 qdisc del dev ath0 root 2> /dev/null
tc2 qdisc add dev ath0 handle 1:0 root hfsc
tc2 filter add dev ath0 parent 1:0 protocol ip prio 6 handle 100 fw flowid 1:1 action mirred egress redirect dev ifb0

iptables -t mangle -A POSTROUTING ! -s 192.168.0.0/16 -j MARK --set-mark 100

ifconfig ifb0 up
tc qdisc del dev ifb0 root 2> /dev/null
tc qdisc add dev ifb0 root handle 1: tbf rate 1300kbit burst 8192b lat 10ms

You'll know if it worked because your downloads will suddenly be slow like mine. After you've confirmed it works, start playing with the 'rate'. I find 80% of the maximum possible throughput works pretty well.

 

[ich.hasse.o2]

Finally it works! Thank you very much.

 

[Sania]

modprobe ifb
echo disable > /proc/net/avm_pa/control

tc2 qdisc del dev cpmac0 root 2> /dev/null
tc2 qdisc add dev cpmac0 handle 1:0 root hfsc
tc2 filter add dev cpmac0 parent 1:0 protocol ip prio 6 handle 100 fw flowid 1:1 action mirred egress redirect dev ifb0

tc2 qdisc del dev ath0 root 2> /dev/null
tc2 qdisc add dev ath0 handle 1:0 root hfsc
tc2 filter add dev ath0 parent 1:0 protocol ip prio 6 handle 100 fw flowid 1:1 action mirred egress redirect dev ifb0

iptables -t mangle -A POSTROUTING ! -s 192.168.0.0/16 -j MARK --set-mark 100

ifconfig ifb0 up
tc qdisc del dev ifb0 root 2> /dev/null
tc qdisc add dev ifb0 root handle 1: tbf rate 1300kbit burst 8192b lat 10ms

Hello to everyone!

I used the above solution for traffic shaping, however it shapes ONLY the IPv4 traffic.
In my case, I have also a native IPv6, which is not affected by the above traffic shaping rules!

Please, can someone tell me what changes should I do in the above commands, or at least give a hint? In order to make the traffic shaping for both IPv4 and IPv6.

Thanks!