AVM's DHCP-server gibt alte WiFi-clients nicht frei

[frater]

It was in no way intended to offend you or saying I was given bad advice.
I'm sure these changes could be done without starting the box.
I don't know for sure these 2 boxes are reset to factory default. They did in similar circumstances in the past several times and never when taking the approach I'm now taking.
Restarting on Monday at 5am only when there are many of these lame clients is unlikely to be inconvenient.

I do agree it is more elegant to only change the appropriate services.
I also agree I shouldn't change my original post at this stage.

The script that reset these boxes is the same a posted with these commands commented out.

Maybe I didn't follow your instructions properly. Could you maybe rewrite these last lines. Do understand that I'm somewhat wary to implement it on a production box.


I have boxes running for a long time that are as responsive as recently started. These boxes with a lot of lame clients are extremely slow wether they are running for a while or just started. These boxes will only become properly responsive when after I remove these clients. If I do this by hand it takes Moe than an hour.

Do bear in mind I'm running many boxes over more than 2 years. You could add another 7 years for DD-WRT.

 

[PeterPawn]

It was in no way intended to offend you or saying I was given bad advice.

I'm able to distinguish between an offence and a difference in opinions and so I'm not taking offence from your opinion.

My wish was only to clarify, that - beside your own experiences - it's very unusual, if the firmware resets a configuration to factory settings without any reason ... no matter, if it's done while executing "rc.net reload" or regular startup.

You were rushing ahead my comment and wanted to make plain, that it could not be your mistake.

I'll accept such a statement, if it's proved or at least reasoned. Please bear in mind, that it seems to be something like a "private conversation", what we're doing here ... but somebody, who's looking for an own solution later and reading this thread, will be entitled to his own opinion only by the writings here. And if there's a statement: "Changing ar7.cfg and restarting ctlmgr leads to factory reset." (it's a little bit exaggregated, but imho the excerpt from #19), he could tend to the wrong solution, that a restart is necessary in each case. And that's false at first and - more important - unwanted and/or impossible and/or unnecessary under many circumstances. That's why I'd like to distinguish between a personal opinion/assumption/report (ar7cfgchanged resets to factory settings) and a proven finding (do step a, b, c to reproduce the behaviour). If you want to discover the reported issue, store - as mentioned before - the new ar7.cfg for later evaluation and double-check after an unexpected factory reset, if there's no own mistake indeed.

I don't know for sure these 2 boxes are reset to factory default. They did in similar circumstances in the past several times and never when taking the approach I'm now taking.

Reread your statement at #19 and tell me, where I've misunderstood what you wrote (the exaggregation was admitted above already). Beside any earlier findings ... it sounds like a repeatable behaviour ? Why don't you publish the changed files (you've written, it was the result of an 'ar7cfgchanged' call, therefore I'd make the conclusion, there was a change to the file prior) ? My question, why the firmware should reject a valid settings file, is even unanswered ... everybody can make a mistake, no matter of his experiences/knowledge and I can't see, why that could not happen to you too. And I - personally - disagree with guessworks without any reasoning ... if you're able to justify the opinion, do it. If you're unable, demarcate between opinions and findings ... that will make it easier for others to track the findings and to present a solution or fix.

Do bear in mind I'm running many boxes over more than 2 years. You could add another 7 years for DD-WRT.

That can't be a reason to believe, you're unable to make mistakes ... but that's not the question.

And to stay in touch with the word "question" ... my only question in #11 was, why you've decided to restart the box after the changes. Even the answer "to restore it to a well-known state" would be a good reason ... that's what I wrote in #16 and there's the first advice too, that I want to express my opinion as a counterpart for later readers.

And please believe, it's not my intention to insult you ... I try as hard as possible to write unobstrusive. But still with respect to your own experiences and those of the vast majority of Freetz users ... and your last sentence in #18 shows, you think too, it's a bit fishy. And while your use case (delete unused clients) may be special, changing the ar7.cfg file of a running system is an every day used solution for many other users - most times without any hassle, if it's done the right way.

EDIT: I've read your changes to #21 now ... if you're okay with a regular restart and can do this while your access points are unused, that may be the right solution for you and I would not change it, if I were in your spot.

 

[frater]

I have carefully examined the difference between ar7.cfg and the changed one with a comparison tool on the box I was testing this script.
I didn't do this on the 2 boxes that were gone. So I don't claim it was.
I hope I can take the time to properly test things on a local box and I will also wrote some extra stuff so I have something to investigate when things go wrong.

 

[frater]

Last Monday was hectic...
I had to go to 4 clients as the "restart" procedure failed on 2 of the 4 other clients.

I have now totally abandoned the approach of modifying ar7.cfg and multid.leases.

In Zabbix I created these 2 items:

UserParameter=net.arpcount, arp 2>/dev/null | grep -c : 
UserParameter=net.leases, grep -c : /var/flash/multid.leases 2>/dev/null

This way I can let Zabbix tell me which clients have more than a 100 lame clients.
I will then remove these by hand (although this takes ages)

 

[PeterPawn]

I have now totally abandoned the approach of modifying ar7.cfg and multid.leases.

I'm sure, there will be good reasons for your decision and even if I'm snoopy, that's not the cause for my next question ...

I will then remove these by hand (although this takes ages)

Did you consider to detect the unused leases as before and automate the remove process (with ctlmgr_ctl or luavar calls) only ? That could be another approach to save your time (even if I can't see, why the first solution is not working, but that's something completely different). A cron job, which is executed hourly and removes the unused entries, is a another possible solution ... and - remember, it's no offense - if something fails on your devices in that regard, there's in my opinion more likely another modification the root cause and not your attempts to remove the unused clients. If you can do it manually without side effects, it can be automated.

That was the objective part, but I'm dwelling on some more thoughts ... they aren't without judgement, but nevertheless I'd like to write them down here (and I'll try to restart my brain afterwards for a fresh start reading your postings in the future).

Of course I'm sure, that your devices never made problems before and the reason for your new problems can't be a/the new firmware version ... at all it worked over years now and you didn't change anything. Only the firmware has been changed and that you took into account already, didn't you ?

You wrote about being wary to change your installation ... why did you do it in the end ?

If you update to a newer version (it's my conclusion from other threads, that you've tried to update at least), you've to expect anytime, that you may face new problems.

And if you're responsible for more than 80 devices, you should consider this every time, if you change something ... therefore I can't understand your anger, which I've read (or even I thought to read) interline in #19 and other postings.

You know, there are some deep changes done by AVM ... if you read some tickets from the Freetz trac system, you'll see, there are far more additional problems with 06.20 versions waiting for a solution.

Maybe that I'm too questioning and the final reason for the tiff is the communication in a foreign language (on both sides) ... but my impression is, you're angry that your installation is not working as expected and you assume any responsibility of Freetz or the Freetz makers to find a solution. If I'm wrong, I'd like to apologize ... but some of your postings (not only in this thread) lead to that feeling. And most times you need some endurance to track down a problem to the roots ... if you raise an issue and someone takes the time to understand your problem and to propose a possible solution, it's unhelpful to quit on first problem setback. If you do this one time, it may be an exception ... but next time your readers could also ignore a new question, if they've only a faint hope to come to a final resolution (and a surrender is frustrating for the helpers too).

 

[frater]

Hi Peter,

No I'm not angry at all.

I still think that keeping info about all clients is wrong. Especially if it bogs down the performance of the box. I have never encountered any dhcp-server that does this.

Now I'm monitoring this I'm seeing more boxes with many lame clients. All these have a slow interface.

I've tested 6.20 on a very small case and because I wanted to implement this script as a binary in the firmware I also pushed the 6.20 firmware on these boxes that turned out to be troublesome. I agree with you that's not wise. I know you shouldn't change more than 1 thing at a time if you need to know what's causing problems.

Most of my boxes are running the same firmware. This is something that's being monitored by zabbix. No box can become unavailable without me knowing exactly when that happened. Also configuration changes are kept and I can revert to any configuration. Even those of a year ago.

I will now take it slow. The boxes that are running 6.2 will stay on that firmware. I will now do these 10 boxes with more than 50 lame leases by hand and will keep them monitored to find out how quickly this goes up.

I'm really not the type of person you think I am. I sometimes take a risk but when it bites me I'm not starting to cry. I know I cut some corners.

 

[PeterPawn]

Hi,

everything is good ...

As I wrote, I'm able to ignore previous tiffs and I'm willing to take a fresh start again.

Now, after you've outlined some backgrounds, I can appreciate some things better ... and if you face additional problems and want to track down the causes, we should be able to support you. And sorting out the issues without eagerness and step by step should even help to make Freetz and the packages better.

With this in mind let's continue ... if you've closed this case, with the next problem.

One last thought about the "misbehaviour" of the DHCP server ... in my opinion there are some changes by AVM (with regards to a better child protection, I assume), that overstated the case. If no (ever seen) network clients are removed anymore, it's usually no problem to the limited extend of a "private installation" and your use case is something very special. But there should be a good probability, that AVM is able to prove the issue and provide (in the next version of the firmware) a "flush" button for older leases without the needs to remove them one by one. But you have to report the problem to AVM too, did you do this already ?

 

[frater]

When something goes wrong with a device that's used by many I always wonder if there is something special with my case. Why is it happening "to me" and not with all those other people. I have seen that large list many times and although I always had the urge to delete some of them I never thought it would be a problem. Only because AVM would not let that list grow and grow otherwise.

You just mentioned the child protection.
Because I don't need child protection I have removed that from the firmware with "make menuconfig".
Could I also have removed some "garbage collector"?

My "top lame client device" has more than 300 entries in multid.leases.
It's a bar that gives all their clients the password of their WiFi.
Although not many people do this it still isn't THAT unusual.
In the Netherlands there are a few ISP's that give a 7360 with their connection.

I didn't report to AVM.
In the past I made a ticket and was honest about using Freetz. That bug was also present in standard AVM but I wanted to be helpful and gave some extra diagnostic info I could only give when I had Freetz on. They told me to report it on this forum. pfffff
Another bug I posted regarding SNOM phones for which SNOM has a dedicated webpage was handled in a bad way as well...
http://wiki.snom.com/FAQ/How_to_fix_my_Snom_phone's_audio_problems_with_Fritzbox?/de

Edit: The bug they rejected because I was using Freetz was about the SIP-client in the Fritzbox. It wasn't able to reach a SIP-server on the LAN anymore after a firmware upgrade. The SIP-server on the LAN was a GSM-gateway to be used for dialing out to mobile phones. I gave some diagnostic info so they could find that bug they introduced a bit quicker. They were doing structural changes to the SIP-framework at the time.
I don't know if that bug was ever fixed. I just downgraded....

 

[PeterPawn]

You just mentioned the child protection.
Because I don't need child protection I have removed that from the firmware with "make menuconfig".
Could I also have removed some "garbage collector"?

Imho that's possible, indeed ...

Why did you remove the child protection ? Only due to lack of space ? Which firmware version and model did/do you use ?

If you really have to remove parts of the original firmware to make room for own extensions, it's - mostly - better to switch to "externals" mode and keep as much as possible of the original parts of the firmware.

If you do not need Windows network connectivity, you can remove the Samba and (if needed) WebDAV functions with very little risks.

Even removing unused language databases (AVM has "reanimated" an older plugin mechanism (with some additions) for the 7390i to "externalize" the language databases) and provider settings (from /etc/default.${CONFIG_PRODUKT}/providers-0xx.tar) could save some space, but you have to do this manually until now.

With new versions of the firmware almost every time new dependencies are introduced and each removal patch is risky without exact knowledge of the consequences for the current version. Therefore using the trunk is almost always only a bet for a full-functioning firmware replacement and removing firmware parts without pressure should be avoided ... but sometimes it's impossible to respect this rule, I know that. If you have a look at the trunk sources, you will see, that the last change to 310-remove-kids.sh was done 13 month ago ... you can be sure, that no-one ever reviewed this remove patch for the new version. If it's functioning, there's no reason to do so ... if it's the cause of your problem, you're probably the first one, who made the point of that.

If you'd done further investigations (e.g. tested a version without removing child protection and the same findings in the end) and can link the missing "garbage collection" for DHCP leases to the remove patch, you should report it (please only, if it's proven and not as an assumption as matters stand now) to the Freetz Trac system.

 

[frater]

Most boxes are/were running 6.03-11878 on 7390i boxes. I've seen these long lists for a long time.
To be honest I'm not getting rid of "kids" to free up resources. I really don't want it in there, just like tr069 which I suspect to cause security issues.

I don't think this problem is introduced recently. I had reports of wlan connections not being able to connect but only now I'm able to link these to those lame clients.

I hope I can find time this week for this. Thanks for hanging in there

 

[PeterPawn]

I spent many hours investigating AVM's firmware and looking for security flaws or examining dependencies between the different parts.

While it's clear and proven, that using a predefined provider setting may overrule (for some providers in Germany only, as far as I found) user's choice to disable TR-069 completely, I would state too, that using the "Anderer Anbieter" setting (no idea, how it's called in English and I can't be bothered to look for it now) will switch off using TR-069 reliably.

The child protection feature was involved into the DHCP server some times ago, afaik (better: as far as I remember) when the internet time budgets and time-of-day restrictions were introduced. The correlation between a device and an access profile is built on top of the IP address (you can find them within the /var/flash/user.cfg file (section users2), even without the assignment of a "fixed address" to the client) and therefore the DHCP server has started to maintain some assignments longer than usual to protect against exploits of it's "dark oblivion".

If you set the "child protection" (it's more likely an access control system nowadays) settings to be harmless and unobstrusive, there's nothing to fear from this feature in my opinion.

 

[frater]

I've been busy with this during the holidays, but didn't want to post each time to prevent giving mixed messages as I did before.
The things I want are working now.

What happened is that I carefully examined if the output of the modified ar7.cfg was completely balanced.
I then made a small modification without testing afterwards.
This "small modification" contained an error, but because the routers had a factory reset afterward I wasn't able to check it...

I've tested with one connection for a while and already felt comfortable enough to replace the firmware of 10 7390's.
I also removed the "kids patch" to check if this behaviour is caused by this patch, but it doesn't...
These entries keep filling up until no IP's can be distributed anymore...

The error I made came from using 'tail -n+1' instead of 'tail -n+2'
Its counterpart with "head" is 'head -n-1'
I wonder if I make that mistake ever again...

So, here's the script that's working and removes lame clients and preserving those that are connected or have some special flags.

$ cat ./make/zabbix_agentd-support/files/root/sbin/tidy_ar7

#!/bin/sh

# A script to get rid of unused leases that clog your AVM Fritzbox when used as a semi-public WiFi
# It is intended to run as a weekly cronjob.
# Something like this
#
# 47 4    * * 7   root    /sbin/tidy_ar7
#

[ -e /var/flash/ar7.cfg ] || exit 1

# Use the arp table to get the currently connected clients
arp | egrep -o '([[:xdigit:]]{2}[:-]){5}[[:xdigit:]]{2}' >/tmp/arplist

LEASES=`grep -c ':' /var/flash/multid.leases`
LAME_LEASES=$(( ${LEASES} - `grep -c ':' /tmp/arplist` ))

if ! egrep -q '^ +landevices \{' /var/flash/ar7.cfg ; then
  echo "/var/flash/ar7.cfg has been changed by AVM and this script is now unable to work properly" >&2
  exit 1
fi

if [ ${LAME_LEASES} -lt 10 ] ; then
  echo "There are only ${LAME_LEASES} lame leases, I will make no change" >&2
  exit 1
fi

# use /var/flash/ar7.cfg to create 3 pieces
egrep -B9999 '^ +landevices \{' /var/flash/ar7.cfg                                     >/tmp/ar7.first
egrep -A9999 '^ +landevices \{' /var/flash/ar7.cfg | egrep -B9999 -m1 '^}' | tail -n+2 >/tmp/ar7.cfg.leases
egrep -A9999 '^ +landevices \{' /var/flash/ar7.cfg | egrep -A9999     '^}'             >/tmp/ar7.last

# Initialize
echo -n '' >/tmp/ar7.middle1
echo -n '' >/tmp/lease
KEEP=
IFS=''

# Parse the leases
while read line ; do
  # We reached the end... exit the loop
  echo "${line}" | egrep -q '^}$' && break

  # Determine if we keep the entry
  echo "${line}" | grep -qif /tmp/arplist  && KEEP=1
  echo "${line}" | grep -qi 'yes'          && KEEP=1
  echo "${line}" | grep -qi 'forwardrules' && KEEP=1
  echo "${line}" | grep -qi ' name ='      && KEEP=1

  # write one line of that section
  echo "${line}" >>/tmp/lease

  if echo "${line}" | egrep -q '}' ; then
    [ ${KEEP} ] && cat /tmp/lease >>/tmp/ar7.middle1
    KEEP=
    echo -n '' >/tmp/lease
  fi

done </tmp/ar7.cfg.leases

# sanity check: last line needs to contain a closing bracket....  otherwise abort
tail -n1 /tmp/ar7.middle1 | egrep -q '^ +}' || exit 1

# last line has either a closing and opening bracket or only a closing bracket
# replace last line with a line with only a closing bracket
head -n-1 /tmp/ar7.middle1  >/tmp/ar7.middle2
echo -e '        }'        >>/tmp/ar7.middle2

# create the new ar7.cfg by sticking the 2 ends to it
cat /tmp/ar7.first    >/tmp/ar7.cfg.new
cat /tmp/ar7.middle2 >>/tmp/ar7.cfg.new
cat /tmp/ar7.last    >>/tmp/ar7.cfg.new

# reminder we really eeed some more sanity checks here....
#
#

# let's do it
# stop the multi-daemon, empty its list and overwrite /var/flash/ar7.cfg
# Then exit with errorlevel 0

# Backup current ar7 and save a copy of the modified one
cat /var/flash/ar7.cfg >/var/flash/ar7.cfg.org
cat /tmp/ar7.cfg.new   >/var/flash/ar7.cfg.modified

# Stop the 2 services involved
/usr/bin/ctlmgr -s
/etc/init.d/rc.multid stop

# replace ar7.cfg with the modified one
cat /tmp/ar7.cfg.new >/var/flash/ar7.cfg
# replace multid.leases
grep  -if /tmp/arplist /var/flash/multid.leases >/tmp/multid.leases
cat /tmp/multid.leases >/var/flash/multid.leases

# Start the 2 services involved
/usr/bin/ctlmgr
/etc/init.d/rc.multid start

# clean the room after playing
rm /tmp/multid.leases
rm /tmp/ar7.first
rm /tmp/ar7.middle1
rm /tmp/ar7.middle2
rm /tmp/ar7.last
rm /tmp/lease
rm /tmp/arplist

exit 0

 

[PeterPawn]

Thanks for your feedback.

As far as I understood, your script is working now without the need to restart the box afterwards, congratulations for solving your problems.

One last question ... I've never checked it myself, it's only based on a theory:

What happens, if a WLAN client is connected, but did not communicate within the last ARP timeout interval ? Each ARP table entry has its own limited lifetime and will expire after it's reached. The management and garbage collection for known neighbors is a complicated process with more than two states (known or not known), an entry may become "stale" and may be renewed by more than an answer to an ARP request too. As far as I know, the ARP lookup table is only "one window" into the neighborhood cache.

Another one is the output of the various (read-only) "files" within the /proc/kdsld/neigh/...-Interface, but due to my own investigations it only lists the active neighbors.

AVM has its own utility to show the known neighbors, together with a relative time value for the last contact. Several contacts may have occured with different addresses and this tool shows even entries, which are really (and I mean really) old (> 90 days).

My consideration is, that there's a possibility to remove an active client, if it does not hold an open connection to a service at the internet (like most tablets/smartphones with "push service" do) and does not request data regularly. You could remove such clients too early. It will not harm any client in your installation, because you remove them only once per week (and - as I assume - while the router is rather unsed), but if someone wants to adopt your script to do some other actions only based on ARP entries, it could matter there.

As I wrote, only some thoughts, solely theoretically and not investigated in that context ... and nothing what should be considered in your installation (as far as I understood, what you're doing there).

 

[frater]

Hi Peter,

As for a client being removed from ar7.cfg or multid.leases this is no problem at all.
The Fritzbox is still a Linux router that will accept incoming traffic on its LAN subnet and route it to the Internet.
No connection is actively denied.
It will get a new entry in this file and I've already see that happen.
Traffic on the LAN is handled by the switch.

Some connections to the outside world (typically SSH terminal sessions) can even survive a reboot of the router as long as no data is getting transferred during the reboot.
I was quite surprised when I saw that happen for the first time.

The only thing that could happen is a new client that will get an IP from a lease that was taken by such a dormant DHCP-client.
This is very very unlikely but is still no big deal. An IP-conflict will be the result and get automatically resolved by a release/request of the DHCP-client.

A dormant DHCP-client that has been removed will over time request its own IP again and will get a new entry in multid.leases and ar7.cfg

 

[frater]

I had 2 boxes returning to factory config again...
I think I will make an alternate version of the script that will save an alternate time tagged config without getting used.

After collecting enough configs I can examine these

 

[zoranm1976]

Please can you explain how to do it.
I have also a lot of fritz boxes and one is running in friends coffie shop. Problem is tha there is 100+ guests with mobile phones and I can't renew leases. Fritz is 7240 but I also have 7270 on hand to use it. Network for customers is closed with key and it is guest network. Fritz is connected to cable modem via LAN1.

 

[frater]

Please can you explain how to do it.
I have also a lot of fritz boxes and one is running in friends coffie shop. Problem is tha there is 100+ guests with mobile phones and I can't renew leases. Fritz is 7240 but I also have 7270 on hand to use it. Network for customers is closed with key and it is guest network. Fritz is connected to cable modem via LAN1.

Please post a report to AVM...
I don't understand at all why this anomaly is there for such a long time.

In modern firmwares you can lower the lease-time to 1 day, which seems to have helped a bit

I just ran my script on a Fritzbox that had more than 400 leases in its /23 network.
It went well, but it isn't solid enough to let it run on a weekly base....

 

[zoranm1976]

Please post a report to AVM...
I don't understand at all why this anomaly is there for such a long time.

In modern firmwares you can lower the lease-time to 1 day, which seems to have helped a bit

I just ran my script on a Fritzbox that had more than 400 leases in its /23 network.
It went well, but it isn't solid enough to let it run on a weekly base....

In crappy modem-router like casda or zyxel you can make leases in seconds I think default is 3600 seconds lease.
What is my main problem that I have a lot of leases from guest access. Main network access is for known peopele and there is 5 phones ad 2 pc-s in it.

 

[frater]

In crappy modem-router like casda or zyxel you can make leases in seconds I think default is 3600 seconds lease.
What is my main problem that I have a lot of leases from guest access. Main network access is for known peopele and there is 5 phones ad 2 pc-s in it.

It slows down the AVM box. Eventually becoming unstable.
Both networks can overflow.

 

[frater]

A week ago I enabled the "cleaning cronjob" on one of the routers that has this problem with too many DHCP-leases.
I have about 5 of them.
Because it worked I decided yesterday evening to enable this "morning cronjob" on another one.
This resulted in 2 routers offline this morning....

I still have to go there and probably both routers restored to factory defaults like they did before.


I would really like to get some help trying to implement a different approach.
I don't want to modify the ar7.cfg & multid.leases anymore.

Like PeterPawn suggested I think it's best to have a cronjob remove the leases using a lua-script in the same way as the AVM WebIF does.
It should only delete entries that don't exist in the ARP-table, but they should exclude those with reserved leases and portforwarding.

The problem for me is that I have no clue at all where to begin.

Could someone please write some code to remove an entry in LUA?
I will promise to use that code as an example to write more code myself and contribute new code to this project. (teach a man to fish.....)
I have already contributed other code, btw

My consideration is, that there's a possibility to remove an active client, if it does not hold an open connection to a service at the internet (like most tablets/smartphones with "push service" do) and does not request data regularly. You could remove such clients too early.

A trick to prevent this from happening is to ping the IP (1 is enough). Even if the firewall is blocking it this will result in an arp-entry.
I use this trick to make sure there is no device on a certain IP.

All entries in the multid.leases could be pinged before the ARP-table is checked.