Automatic backup of config using a "pull method"

frater · 11 Mai 2013

There is already a thread about an automated backup using a cronjob on the Freetz-box.
But I would like to get some pointers on how to write an automated script on a linux server that's pulling all the configs..

I wrote such a script in the past and it was even able to do a mixed set of routers (netscreen, cisco, siemens and speedtouch)....
Every brand had its specialized method of obtaining a config.

I would like to write such a script again, but then for freetz
I can do the whole thing, but I do need to know what to do to get the actual config (using curl or wget)

This is how I do it for my pfsense router...

Code:

  wget -qO/dev/null --keep-session-cookies --save-cookies /tmp/pfsense_cookies.txt  --post-data "login=Login&usernamefld=${USER}&passwordfld=${PASS}" ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
  wget -qO${FNAME} --keep-session-cookies --load-cookies /tmp/pfsense_cookies.txt  --post-data 'Submit=download&donotbackuprrd=yes' ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php

I would really like to know how to do something similar on a Freetz router.
Maybe even using the AVM-interface (or both)....

The whole cronjob looks like this (most code is about getting rid of the config files that are (almost) the same as the previous one):

Code:

#!/bin/sh

DATESTAMP=`date +%Y-%m-%d.%H:%M`
FNAME=pfsense.${DATESTAMP}.xml
FOLDER=/var/www/vhosts/domain.com/pfsense

USER=admin
PASS=secret

PROTO=https             # http or https
IP=8.8.8.8                 # DNS or IP of webif (remote side)
PORT=9180                # port of webif (remote side)
WGETOPT=

# turn off certificate checking
[ "${PROTO}" = "https" ] && WGETOPT="${WGETOPT} --no-check-certificate"

if cd ${FOLDER} ; then

  FGROUP=`stat -c%G .`
  FUSER=`stat -c%U .`
  LASTXML=`ls -1t pfsense*xml 2>/dev/null | head -n1`

  wget -qO/dev/null --keep-session-cookies --save-cookies /tmp/pfsense_cookies.txt  --post-data "login=Login&usernamefld=${USER}&passwordfld=${PASS}" ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php
  wget -qO${FNAME} --keep-session-cookies --load-cookies /tmp/pfsense_cookies.txt  --post-data 'Submit=download&donotbackuprrd=yes' ${WGETOPT} ${PROTO}://${IP}:${PORT}/diag_backup.php

  if [ -s ${FNAME} ] ; then
    chown ${FUSER}:${FGROUP} ${FNAME}

    if [ ! -z "${LASTXML}" ] ; then
      if [ ! "${LASTXML}" = "${FNAME}" ] ; then
        echo '<time>[0-9]+</time>'       >/tmp/pfgrep.tmp
        echo 'made unknown change'      >>/tmp/pfgrep.tmp
        echo 'Expired .* user accounts' >>/tmp/pfgrep.tmp

        egrep -vf /tmp/pfgrep.tmp ${LASTXML} >${LASTXML}.notime
        egrep -vf /tmp/pfgrep.tmp ${FNAME}   >${FNAME}.notime

        if diff ${LASTXML}.notime ${FNAME}.notime >${FNAME}.diff ; then
          rm -f ${FNAME}*
        else
          chown ${FUSER}:${FGROUP} ${FNAME}.diff
        fi

        rm /tmp/pfgrep.tmp
        rm -f ${LASTXML}.notime 2>/dev/null
        rm -f   ${FNAME}.notime 2>/dev/null
      fi
    fi
  else
    rm -f ${FNAME}
    exit 1
  fi
else
  exit 1
fi

olistudent · 11 Mai 2013

What about tracing a Freetz backup call with Firebug etc. This should reveal the URL and parameters to call...

Regards
Oliver

MaxMuster · 11 Mai 2013

Code:

wget --http-user=admin --http-password=ganzgeheim http://fritz.box:81/cgi-bin/backup/do_backup.cgi -q -O BU_$(date +'%Y%m%d').tgz

frater · 11 Mai 2013

MaxMuster schrieb:

Thank you!!!

I will come back to this thread after I made a portable script that can be used by others.....
One will end up with a folder with all the configs sorted....

frater · 11 Mai 2013

I wrote this cronjob which uses a .list file containing the IP-addresses of the modems it should fetch the config from.
That file is read from the same folder where also the configs are placed.

It's not working as it should because each time the config is download I will get a different file although the config hasn't changed.
There's probably a good reason for this.

I wanted to investigate it, so I did a "tar xvzf config.tgz", but that gives me an error.
Is this normal?

Here's the script...
I will improve it somewhat... especially adding some decent logging

root@ns5:~# cat /usr/local/sbin/freetz_getconfigs

Code:

#!/bin/sh

DATESTAMP=`date +%Y-%m-%d.%H:%M`
FNAME=pfsense.${DATESTAMP}.xml
FOLDER=/var/www/vhosts/domain.nl/freetz

USER=admin
PASS_DEFAULT=secret
PORT_DEFAULT=6080

if cd ${FOLDER} ; then

  FGROUP=`stat -c%G .`
  FUSER=`stat -c%U .`

  while read MODEMLINE ; do
    IP="`echo "${MODEMLINE}" | awk -F, '{print $1}'`"
    PASS="`echo "${MODEMLINE}" | awk -F, '{print $2}'`"
    PORT="`echo "${MODEMLINE}" | awk -F, '{print $3}' | tr -cd '0-9'`"

    if [ -n "${IP}" ] ; then
      [ -z "${PORT}" ] && PORT=${PORT_DEFAULT}
      [ -z "${PASS}" ] && PASS=${PASS_DEFAULT}

      FNAME=${IP}.${DATESTAMP}.tgz

      if wget --http-user=admin --http-password=${PASS} http://${IP}:${PORT}/cgi-bin/backup/do_backup.cgi -q -O ${FNAME} 2>/dev/null ; then
         if [ -s ${FNAME} ] ; then
           chown ${FUSER}:${FGROUP} ${FNAME}

           mkdir ${IP} 2>/dev/null && chown ${FUSER}:${FGROUP} ${IP}

           LATESTCONFIG=`find "${FOLDER}/${IP}" -maxdepth 1 -type f -size +20000 | grep "${IP}.*\.tgz" | xargs -I{} stat -c '%Y %n' {} | sort -rn | head -n1 | awk '{print $2}'`
           if [ -z "${LATESTCONFIG}" ] ; then
             mv ${FNAME} ${IP}/
           else
             if diff ${FNAME} ${LATESTCONFIG} 2>/dev/null ; then
               mv ${FNAME} ${IP}/
             else
               rm -f ${FNAME}
             fi
           fi
           cd ${FOLDER}
         fi
      else
        echo "Error getting config from ${IP}" 2>/dev/null
      fi
    fi
  done<${0//*\/}.list
else
  echo "${FOLDER} does not exist" >&2
  exit 1
fi

MaxMuster · 12 Mai 2013

What if you do this command alone, without -q ? For me it fails one time and tries again. The second time I get the desired file. It might be easier/better, but at least it worked for me...

vice_pres · 12 Mai 2013

I think it would be even more interesting with backing up the fritzbox config itself (the .export file) too - I do it on some freetz boxes with callmonitor described here (http://www.ip-phone-forum.de/showthread.php?t=257074) - but there are 2 Boxes i would like to get the .export from that don't have freetz for a reason... Maybe including the .export in your script would be interessting for you, too.

frater · 12 Mai 2013

This script is working.....

It needs a file "freetz_getconfigs.list" in the folder where you want the configs (/var/www/vhosts/domain.com/freetz)

Each row in that file is a freetz modem with a comma between each parameter....

<HOST>,<IP>,<PORT>,<PASSWORD>

I installed it as nightly cronjob

UPDATED ON: 13-5-2013 13:44

root@ns5:# cat `which freetz_getconfigs`

Code:

#!/bin/sh

DATESTAMP=`date +%Y-%m-%d.%H-%M`
FNAME=pfsense.${DATESTAMP}.xml
FOLDER=/var/www/vhosts/domain.com/freetz

USER=admin
PASS_DEFAULT=secret
PORT_DEFAULT=6080
PROTO=http
MINSIZE=20000

WGETOPT=

patch_files ()
{
  # remove or patch some of the untarred files as they spoil the check for a changed config
  if cd "$1" >/dev/null 2>&1 ; then
    rm -f stat.cfg chrony.drift multid.leases 2>/dev/null
    rm -f fx_cg fonctrl phonebook voipd_call_stat 2>/dev/null

    sed -i '/[0-9][0-9]:[0-9][0-9]:[0-9][0-9]/d' voip.cfg   # remove datestamp in voip.cfg
    sed -i 's/ *\/\*AVM\*\///g' ar7.cfg                     # remove the /*AVM*/ tag in ar7.cfg
    sed -i 's/; *$/;/g' ar7.cfg                             # remove trailing spaces after a semicolon
    cd - >/dev/null 2>&1
  fi
}


if cd ${FOLDER} ; then

  FGROUP=`stat -c%G .`
  FUSER=`stat -c%U .`

  while read MODEMLINE ; do
    HOST="`echo "${MODEMLINE}" | awk -F, '{print $1}'`"
    IP="`echo "${MODEMLINE}"   | awk -F, '{print $2}'`"
    PASS="`echo "${MODEMLINE}" | awk -F, '{print $3}'`"
    PORT="`echo "${MODEMLINE}" | awk -F, '{print $4}' | tr -cd '0-9'`"

    if [ -n "${HOST}" ] && ! echo "${HOST}" | grep -q '^#' ; then
      if [ -z "${IP}" ] ; then
        IP=`host -t A ${HOST} 2>/dev/null | grep -o 'has address .*' | awk '{print $3}' | head -n1`
      fi
      if [ -z "${IP}" ] ; then
        echo "Unable to resolve ${HOST}" >&2
      else
        [ -z "${PORT}" ] && PORT=${PORT_DEFAULT}
        [ -z "${PASS}" ] && PASS=${PASS_DEFAULT}

        FNAME=${IP}.${DATESTAMP}.tgz
        touch "${FNAME}"
        wget -q --timeout=6 --http-user=admin --http-password=${PASS} http://${IP}:${PORT}/cgi-bin/backup/do_backup.cgi -O "${FNAME}" 2>/dev/null &
        n=1
        while sleep 1 ; do
          [ `stat -c%s "${FNAME}"` -ge ${MINSIZE} ] && break
          [ $n -gt 7 ] && break
          let n+=1
        done
        if [ `stat -c%s "${FNAME}"` -gt 0 ] ; then
          # If I received at least some data, then wait some more and it might get everything
          sleep 2
        else
          echo "wget didn't get any data from host \"${HOST}\"" >&2
          exec 2>/dev/null
          kill -9 %1 2>/dev/null
          exec 2>&2
        fi

        if [ `stat -c%s "${FNAME}"` -lt ${MINSIZE} ] ; then
          echo "Error getting config from ${HOST} on ${IP}" >&2
        else
          chown ${FUSER}:${FGROUP} ${FNAME}
          mkdir ${HOST} 2>/dev/null && chown ${FUSER}:${FGROUP} ${HOST}

          LATESTCONFIG=`find "${FOLDER}/${HOST}" -maxdepth 1 -type f -size +10 | grep "${IP}.*\.tgz" | xargs -I{} stat -c '%Y %n' {} | sort -rn | head -n1 | awk '{print $2}'`
          # If no older config is found or they have exactly the same name
          if [ -z "${LATESTCONFIG}" ] || [ "${LATESTCONFIG//*\/}" = ${FNAME} ]; then
            mv ${FNAME} ${HOST}/
          else
            rm -rf folder1 2>/dev/null ; mkdir folder1
            rm -rf folder2 2>/dev/null ; mkdir folder2
            if ! tar xzf ${FNAME} -C folder1 2>/dev/null ; then
              echo "Invalid gzipped tar-file" >&2
              rm -f ${FNAME}
            else
              tar xzf ${LATESTCONFIG} -C folder2

              # Remove files that are likely to be changed, but don't contain important config
              patch_files folder1/var_flash
              patch_files folder2/var_flash

              if ! diff folder1/var_flash folder2/var_flash >/dev/null 2>&1 ; then
                mv ${FNAME} ${HOST}/
              fi
            fi
            rm -rf folder1 2>/dev/null
            rm -rf folder2 2>/dev/null
          fi
        fi
        cd ${FOLDER}
        rm -f "${FNAME}" 2>/dev/null
      fi
    fi
  done<${0//*\/}.list
else
  echo "${FOLDER} does not exist" >&2
  exit 1
fi

frater · 12 Mai 2013

vice_pres schrieb:
I think it would be even more interesting with backing up the fritzbox config itself (the .export file) too - I do it on some freetz boxes with callmonitor described here (http://www.ip-phone-forum.de/showthread.php?t=257074) - but there are 2 Boxes i would like to get the .export from that don't have freetz for a reason... Maybe including the .export in your script would be interessting for you, too.

Yes, I would like to have that .export as well, but the link you gave refers to a script pushing that file...
I have good reasons to choose for a pull system.

It's the next step.....
I can do it in one script.....

What's the wget?

vice_pres · 13 Mai 2013

Yeah I know it's a push - and I can't push from 2 Modems i want to backup so I'm using for a pull myself. But I didn't find a way working without callmonitor - you have to login to the webinterface to get the .export

frater · 13 Mai 2013

I updated the script.....

BTW.. the error was caused by the colon in the variable DATESTAMP which was used as part of the filename.
I went nuts because fetching the file on the prompt was alright and it wasn't in the script.

The script now seems to work fine....

In the near future it will take parameters so it can fetch the config of only one host.
It will then also be possible to call the script from zabbix....
That way each "zabbix monitored freetz modem" will automatically have their config back-upped and I can also check if this succeeded.....

If a new file is added it will also get a flag that the config has changed.....

It feels so good if you have more grip on things....

frater · 13 Mai 2013

I now have a script I can work with, but a Freetz config doesn't help me when I want to configure a replacement router.
If I load the tgz-file, I can't login afterwards and probably all the passwords are hashed against its internal serial number.

I really need a portable .config file
The one you can download from the AVM-interface using a password.....

I just installed/started firebug and fetched a config-file, but I have no idea how to crawl for my .config file

frater · 13 Mai 2013

I found this: https://home.debian-hell.org/blog/2...vm-fritzbox-7390-per-wgetcurl-script-sichern/
and it's working!

The problem is that it's only working from LAN-side and not from WAN.

I also found this: http://www.autoitscript.com/forum/topic/138989-fb-tools-manage-your-fritzbox-from-autoit/

RalfFriedl · 13 Mai 2013

frater schrieb:
Each row in that file is a freetz modem with a comma between each parameter....

<HOST>,<IP>,<PORT>,<PASSWORD>

You should also make the user name configurable.

vice_pres · 13 Mai 2013

frater schrieb:
I found this: https://home.debian-hell.org/blog/2...vm-fritzbox-7390-per-wgetcurl-script-sichern/
and it's working!

The problem is that it's only working from LAN-side and not from WAN.

Besides the "LAN-side" Problem there is another Problem:
Without password provided for the .config the .export is not portable to another box since it's hashed against the SN or something...

Edit:
Da wir doch scheinbar alle Deutsch sprechen - können wir das nicht auch hier? Ich finds soooviel einfacher

frater · 14 Mai 2013

vice_pres schrieb:
Besides the "LAN-side" Problem there is another Problem:
Without password provided for the .config the .export is not portable to another box since it's hashed against the SN or something...

Edit:
Da wir doch scheinbar alle Deutsch sprechen - können wir das nicht auch hier? Ich finds soooviel einfacher

Although I can speak German, it's not my native tongue and it's much harder for me to express myself.
Especially technical stuff....

Daniel Graf has reacted and published something he says is working.....
https://home.debian-hell.org/blog/2...vm-fritzbox-7390-per-wgetcurl-script-sichern/

I don't think it is, but I'm sure he will get it going....

I will then integrate it in my script and we'll have the Freetz-tgz and an AVM .config which is portable....

frater · 14 Mai 2013

The AVM-part is not working....
I've asked Daniel, the one who wrote the base for it (https://home.debian-hell.org/blog/2...vm-fritzbox-7390-per-wgetcurl-script-sichern/) to have a look at it....

Maybe you can test it on your router and give feedback....
Most of my routers fail the challenge and the rest don't get a valid config....

Code:

#!/bin/sh

DATESTAMP=`date +%Y-%m-%d.%H-%M`
FNAME=
FOLDER=/var/www/vhosts/domain.com/freetz

USER_DEFAULT=admin
PASS_DEFAULT=secret
PORT_DEFAULT=6080
PORTS_DEFAULT=450
MINSIZE=20000

get_avm_config ()
{

TMP1=`mktemp`
URL="https://${IP}:${PORTS}"

# get challenge key from FB
_CHALLENGE=$(curl -s \
                  -k \
                  "${URL}/login.lua" | \
                  grep "^g_challenge" | \
                  awk -F'"' '{ print $2 }')

if [ -z "${_CHALLENGE}" ] ; then
  echo "Error getting challenge for ${HOST} using ${URL}" >&2
else
  # echo "Received Challenge \"${_CHALLENGE}\"" >&2
  # build md5 from challenge key and password
  _MD5=$(echo -n \
            ${_CHALLENGE}"-"${_PASSWORD} | \
        iconv -f ISO8859-1 \
              -t UTF-16LE | \
        md5sum -b | \
        awk '{print substr($0,1,32)}')

  # assemble challenge key and md5
  _RESPONSE=${_CHALLENGE}"-"${_MD5}

  # get sid for later use
  _SID=$(curl -i \
            -s \
            -k \
            -d 'response='${_RESPONSE} \
            -d 'page=' \
            -d 'username='${USER} \
            ${URL}/login.lua | \
        grep "Location:" | \
        awk -F'=' {' print $NF '})

  # get configuration from FB and write to STDOUT
  if ! curl -s \
  -k \
  --form 'sid='${_SID} \
  --form 'ImportExportPassword='${PASS} \
  --form 'ConfigExport=' \
  ${URL}/cgi-bin/firmwarecfg >${TMP1} ; then
    echo "Error getting .config for ${HOST} (no data)" >&2
    rm -f ${TMP1} 2>/dev/null
  else
    if [ `stat -c%s "${TMP1}"` -lt 500 ] ; then
      echo "Error getting .config for ${HOST} using ${URL} (too small)" >&2
      rm -f ${TMP1}
    else
      chown ${FUSER}:${FGROUP} ${TMP1}
      mv ${TMP1} ${HOST}/${IP}.${DATESTAMP}.config
    fi
  fi

fi
}

patch_files ()
{
  # remove or patch some of the untarred files as they spoil the check for a changed config
  if cd "$1" >/dev/null 2>&1 ; then
    rm -f stat.cfg chrony.drift multid.leases 2>/dev/null
    rm -f fx_cg fonctrl phonebook voipd_call_stat 2>/dev/null

    sed -i '/[0-9][0-9]:[0-9][0-9]:[0-9][0-9]/d' voip.cfg   # remove datestamp in voip.cfg
    sed -i 's/ *\/\*AVM\*\///g' ar7.cfg                     # remove the /*AVM*/ tag in ar7.cfg
    sed -i 's/; *$/;/g' ar7.cfg                             # remove trailing spaces after a semicolon
    cd - >/dev/null 2>&1
  fi
}


if cd ${FOLDER} ; then

  FGROUP=`stat -c%G .`
  FUSER=`stat -c%U .`

  while read MODEMLINE ; do
    HOST="`echo "${MODEMLINE}"  | awk -F, '{print $1}'`"
    IP="`echo "${MODEMLINE}"    | awk -F, '{print $2}'`"
    PASS="`echo "${MODEMLINE}"  | awk -F, '{print $3}'`"
    PORT="`echo "${MODEMLINE}"  | awk -F, '{print $4}' | tr -cd '0-9'`"
    USER="`echo "${MODEMLINE}"  | awk -F, '{print $5}'`"
    PORTS="`echo "${MODEMLINE}" | awk -F, '{print $6}' | tr -cd '0-9'`"

    if [ -n "${HOST}" ] && ! echo "${HOST}" | grep -q '^#' ; then
      # echo $HOST
      if [ -z "${IP}" ] ; then
        IP=`host -t A ${HOST} 2>/dev/null | grep -o 'has address .*' | awk '{print $3}' | head -n1`
      fi
      if [ -z "${IP}" ] ; then
        echo "Unable to resolve ${HOST}" >&2
      else
        [ -z "${PORT}" ]  &&  PORT=${PORT_DEFAULT}
        [ -z "${PASS}" ]  &&  PASS=${PASS_DEFAULT}
        [ -z "${USER}" ]  &&  USER=${USER_DEFAULT}
        [ -z "${PORTS}" ] && PORTS=${PORTS_DEFAULT}

        FNAME=${IP}.${DATESTAMP}.tgz
        touch "${FNAME}"
        wget -q --timeout=6 --http-user=admin --http-password=${PASS} http://${IP}:${PORT}/cgi-bin/backup/do_backup.cgi -O "${FNAME}" 2>/dev/null &
        n=1
        while sleep 1 ; do
          [ `stat -c%s "${FNAME}"` -ge ${MINSIZE} ] && break
          [ $n -gt 7 ] && break
          let n+=1
        done
        if [ `stat -c%s "${FNAME}"` -gt 0 ] ; then
          # If I received at least some data, then wait some more and it might get everything
          sleep 2
        else
          echo "wget didn't get any data from host \"${HOST}\"" >&2
          exec 2>/dev/null
          kill -9 %1 2>/dev/null
          exec 2>&2
        fi

        if [ `stat -c%s "${FNAME}"` -lt ${MINSIZE} ] ; then
          echo "Error getting config from ${HOST} on ${IP}" >&2
        else
          chown ${FUSER}:${FGROUP} ${FNAME}
          mkdir ${HOST} 2>/dev/null && chown ${FUSER}:${FGROUP} ${HOST}

          LATESTCONFIG=`find "${FOLDER}/${HOST}" -maxdepth 1 -type f -size +10 | grep "${IP}.*\.tgz" | xargs -I{} stat -c '%Y %n' {} | sort -rn | head -n1 | awk '{print $2}'`
          # If no older config is found or they have exactly the same name
          if [ -z "${LATESTCONFIG}" ] || [ "${LATESTCONFIG//*\/}" = ${FNAME} ]; then
            mv ${FNAME} ${HOST}/
            get_avm_config
          else
            rm -rf folder1 2>/dev/null ; mkdir folder1
            rm -rf folder2 2>/dev/null ; mkdir folder2
            if ! tar xzf ${FNAME} -C folder1 2>/dev/null ; then
              echo "Invalid gzipped tar-file" >&2
              rm -f ${FNAME}
            else
              tar xzf ${LATESTCONFIG} -C folder2

              # Remove files that are likely to be changed, but don't contain important config
              patch_files folder1/var_flash
              patch_files folder2/var_flash

              if ! diff folder1/var_flash folder2/var_flash >/dev/null 2>&1 ; then
                mv ${FNAME} ${HOST}/
                get_avm_config
              elif [ ! -f ${HOST}/*.config ] ; then
                # If no .config exists then fetch it anyhow
                get_avm_config
              fi
            fi
            rm -rf folder1 2>/dev/null
            rm -rf folder2 2>/dev/null
          fi
        fi
        cd ${FOLDER}
        rm -f "${FNAME}" 2>/dev/null
      fi
    fi
  done<${0//*\/}.list
else
  echo "${FOLDER} does not exist" >&2
  exit 1
fi

frater · 15 Mai 2013

It is working now!
I'm getting a Freetz config and a portable config from the AVM-interface (with a password)
Thanks to Daniel who figured out how to get the info from the AVM-interface...
I needed to tinker a bit as I'm not getting the same response from every router.
I have more than 20 routers to test and there are 4 models (7270v2, 7270v3, 3270, 7390 and 7360)

Please give me some feedback...
Look at the code to see how you should make the list (which should be in the folder where you want the configs to be stored)

Code:

#!/bin/sh

DATESTAMP=`date +%Y-%m-%d.%H-%M`
FNAME=
FOLDER=/var/www/vhosts/domain.com/freetz

USER_DEFAULT=admin
PASS_DEFAULT=secret
PORT_DEFAULT=6080
PORTS_DEFAULT=450
MINSIZE=20000

get_avm_config ()
{
TMPDIR=`mktemp -p /dev/shm -d ${0//*\/}.XXXXXXXXXX`
if cd ${TMPDIR} ; then

  URL="https://${IP}:${PORTS}"

  SPEED=`httping -c1 -t1 ${URL} 2>/dev/null | \
         egrep '.+/.+/.+/.+/.+' | tail -n1 | \
         awk -F/ '{print $4}' | awk -F. '{print $1}'`

  if [ -z "${SPEED}" ] ; then
    echo "${URL} is unreachable" >&2
  elif [ ${SPEED} -gt 3000 ] ; then
    echo "${URL} is slow (${SPEED} ms)" >&2
  else
  # get challenge key from FB
  _CHALLENGE=`curl -s \
                   -k \
                   --user ${USER}:${PASSE} \
                   "${URL}/login.lua" | \
                   grep 'challenge' | egrep '[0-9a-f]{8}' | \
                   awk -F= '{print $2}' | tail -n1 | \
                   egrep -o '[0-9a-f]{8}'`

  if [ -z "${_CHALLENGE}" ] ; then
    echo "Error getting challenge for ${HOST} using ${URL}" >&2

  else
    # echo "Received Challenge \"${_CHALLENGE}\"" >&2
    # build md5 from challenge key and password
    _MD5=$(echo -n \
            ${_CHALLENGE}"-"${PASSE} | \
        iconv -f ISO8859-1 \
              -t UTF-16LE | \
        md5sum -b | \
        awk '{print substr($0,1,32)}')

    # assemble challenge key and md5
    _RESPONSE=${_CHALLENGE}"-"${_MD5}

    # get sid for later use
    _SID=$(curl -i \
            -s \
            -k \
            --user ${USER}:${PASSE} \
            -d 'response='${_RESPONSE} \
            -d 'page=' \
            -d 'username='${USER} \
            ${URL}/login.lua | \
        grep "Location:" | \
        awk -F'=' {' print $NF '})

    # get configuration from FB and write to STDOUT
    if ! curl -s \
      -k \
      --user ${USER}:${PASSE} \
      --form 'sid='${_SID} \
      --form 'ImportExportPassword='${PASSE} \
      --form 'ConfigExport=' \
      ${URL}/cgi-bin/firmwarecfg >TMP1 ; then
      echo "Error getting .config for ${HOST} (no data)" >&2
    else
      if [ `stat -c%s TMP1` -lt ${MINSIZE} ] ; then
        echo "Error getting .config for ${HOST} using ${URL} (too small)" >&2
        echo "SID: \"${_SID}\" CHALLENGE: \"${_CHALLENGE}\"  MD5: \"${_MD5}\"  RESPONSE: \"${_RESPONSE}\"" >&2
      else
        chown ${FUSER}:${FGROUP} TMP1
        mv TMP1 ${FOLDER}/${HOST}/${IP}.${DATESTAMP}.config
      fi
    fi
  fi
  fi
fi
cd ${FOLDER}
rm -rf ${TMPDIR}
}

patch_files ()
{
  # remove or patch some of the untarred files as they spoil the check for a changed config
  if cd "$1" >/dev/null 2>&1 ; then
    rm -f stat.cfg chrony.drift multid.leases 2>/dev/null
    rm -f fx_cg fonctrl phonebook voipd_call_stat 2>/dev/null

    sed -i '/[0-9][0-9]:[0-9][0-9]:[0-9][0-9]/d' voip.cfg   # remove datestamp in voip.cfg
    sed -i 's/ *\/\*AVM\*\///g' ar7.cfg                     # remove the /*AVM*/ tag in ar7.cfg
    sed -i 's/; *$/;/g' ar7.cfg                             # remove trailing spaces after a semicolon
    cd - >/dev/null 2>&1
  fi
}


if cd ${FOLDER} ; then

  FGROUP=`stat -c%G .`
  FUSER=`stat -c%U .`

  while read MODEMLINE ; do
    HOST="`echo "${MODEMLINE}"  | awk -F, '{print $1}'`"
    IP="`echo "${MODEMLINE}"    | awk -F, '{print $2}'`"
    PASS="`echo "${MODEMLINE}"  | awk -F, '{print $3}'`"
    PORT="`echo "${MODEMLINE}"  | awk -F, '{print $4}' | tr -cd '0-9'`"
    USER="`echo "${MODEMLINE}"  | awk -F, '{print $5}'`"
    PASSE="`echo "${MODEMLINE}" | awk -F, '{print $6}'`"
    PORTS="`echo "${MODEMLINE}" | awk -F, '{print $7}' | tr -cd '0-9'`"

    if [ -n "${HOST}" ] && ! echo "${HOST}" | grep -q '^#' ; then
      echo $HOST
      if [ -z "${IP}" ] ; then
        IP=`host -t A ${HOST} 2>/dev/null | grep -o 'has address .*' | awk '{print $3}' | head -n1`
      fi
      if [ -z "${IP}" ] ; then
        echo "Unable to resolve ${HOST}" >&2
      else
        [ -z "${PORT}" ]  &&  PORT=${PORT_DEFAULT}      # port of Freetz interface
        [ -z "${PASS}" ]  &&  PASS=${PASS_DEFAULT}      # internal password
        [ -z "${USER}" ]  &&  USER=${USER_DEFAULT}      # external username
        [ -z "${PASSE}" ] && PASSE=${PASS}              # external password
        [ -z "${PORTS}" ] && PORTS=${PORTS_DEFAULT}     # port of AVM interface (https)

        FNAME=${IP}.${DATESTAMP}.tgz
        touch "${FNAME}"
        wget -q --timeout=6 --http-user=admin --http-password=${PASS} http://${IP}:${PORT}/cgi-bin/backup/do_backup.cgi -O "${FNAME}" 2>/dev/null &
        n=1
        while sleep 1 ; do
          [ `stat -c%s "${FNAME}"` -ge ${MINSIZE} ] && break
          [ $n -gt 7 ] && break
          let n+=1
        done
        if [ `stat -c%s "${FNAME}"` -gt 0 ] ; then
          # If I received at least some data, then wait some more and it might get everything
          sleep 2
        else
          echo "wget didn't get any data from host \"${HOST}\"" >&2
          exec 2>/dev/null
          kill -9 %1 2>/dev/null
          exec 2>&2
        fi

        if [ `stat -c%s "${FNAME}"` -lt ${MINSIZE} ] ; then
          echo "Error getting config from ${HOST} on ${IP}" >&2
        else
          chown ${FUSER}:${FGROUP} ${FNAME}
          mkdir ${FOLDER}/${HOST} 2>/dev/null && chown ${FUSER}:${FGROUP} ${FOLDER}/${HOST}

          LATESTCONFIG=`find "${FOLDER}/${HOST}" -maxdepth 1 -type f -size +10 | grep "${IP}.*\.tgz" | xargs -I{} stat -c '%Y %n' {} | sort -rn | head -n1 | awk '{print $2}'`
          # If no older config is found or they have exactly the same name
          if [ -z "${LATESTCONFIG}" ] || [ "${LATESTCONFIG//*\/}" = ${FNAME} ]; then
            mv ${FNAME} ${FOLDER}/${HOST}/
            get_avm_config
          else
            rm -rf folder1 2>/dev/null ; mkdir folder1
            rm -rf folder2 2>/dev/null ; mkdir folder2
            if ! tar xzf ${FNAME} -C folder1 2>/dev/null ; then
              echo "Invalid gzipped tar-file" >&2
              rm -f ${FNAME}
            else
              tar xzf ${LATESTCONFIG} -C folder2

              # Remove files that are likely to be changed, but don't contain important config
              patch_files folder1/var_flash
              patch_files folder2/var_flash

              if ! diff folder1/var_flash folder2/var_flash >/dev/null 2>&1 ; then
                mv ${FNAME} ${FOLDER}/${HOST}/
                get_avm_config
              elif [ `find ${FOLDER}/${HOST} -maxdepth 0 -type f -name \*.config | wc -l` -lt 1 ] ; then
                # If no .config exists then fetch it anyhow
                get_avm_config
              fi
            fi
            rm -rf folder1 2>/dev/null
            rm -rf folder2 2>/dev/null
          fi
        fi
        cd ${FOLDER}
        rm -f "${FNAME}" 2>/dev/null
      fi
    fi
  done<${0//*\/}.list
else
  echo "${FOLDER} does not exist" >&2
  exit 1
fi

frater · 16 Mai 2013

I just updated the script to correct a little error (checking if an AVM config has been made at all for a specific host)

I will clean this script up in the near future...
It's a bit of a mess with too many nests in 1 procedure....

This is only to make the script better readable and therefore better to debug.

You need to create a list with all the modems you want to check.
The filename of the list is the same as the name of the script with ".list" attached.

If this list only has a host-address it will use the default values for the Freetz-interface and AVM-interface.
The parameters are separated with comma's

<HOST>,<IP>,<PASSfreetz>,<PORTfreetz>,<USER>,<PASSavm>,<PORTavm>

If the first HOST is resolvable to an IP you don't need to give the IP.
The optional <IP> overrides <HOST>
<HOST> will then only be used as the name of the folder in which the configs will come.

vice_pres · 16 Mai 2013

Your script is working for 2 7270 that I use to pull the config from, didn't test the others since they are pulling. I modified it a bit since the 2 Boxes I need don't have freetz and since the script don't need to try to get the .tgz. The Script from Daniel didn't work for me in the first place so I took your script and ripped the freetz parts out of it

I also inserted a small

Code:

        mkdir -p ${FOLDER}/${HOST}
        chown ${FUSER}:${FGROUP} ${FOLDER}/${HOST}

before moving the .config file since I don't want to create the folder by myself if I add another host

Thanks alot!

Automatic backup of config using a "pull method"

Mitglied

IPPF-Urgestein

IPPF-Promi

Mitglied

Mitglied

IPPF-Promi

Mitglied

Mitglied

Mitglied

Mitglied

Mitglied

Mitglied

Mitglied

IPPF-Urgestein

Mitglied

Mitglied

Mitglied

Mitglied

Mitglied

Mitglied

Statistik des Forums