freeradius memory leak

classic Classic list List threaded Threaded
5 messages Options
| Threaded
Open this post in threaded view
|

freeradius memory leak

Alexander Clouter
Hi,

I have shitty news and really shitty news.

The 'good' news, I know FreeRADIUS (2.1.x branch) has a memory leak.  
The bad news, I'm running in production FreeRADIUS on ARM boxen so
things like valgrind are not an option.  All I see is a SIGKILL.

Spiel from top :)
----
 4260 freerad   20   0  411m 354m 3708 S  4.3 70.4  31:40.31 freeradius
----

What I do have is a 420MB core file of that above process..any takers :)

Ideas welcomed.

Cheers

--
Alexander Clouter
.sigmonster says: For adult education nothing beats children.

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
| Threaded
Open this post in threaded view
|

Re: freeradius memory leak

Alexander Clouter
Alexander Clouter <[hidden email]> wrote:

>
> I have shitty news and really shitty news.
>
> The 'good' news, I know FreeRADIUS (2.1.x branch) has a memory leak.  
> The bad news, I'm running in production FreeRADIUS on ARM boxen so
> things like valgrind are not an option.  All I see is a SIGKILL.
>
> Spiel from top :)
> ----
> 4260 freerad   20   0  411m 354m 3708 S  4.3 70.4  31:40.31 freeradius
> ----
>
> What I do have is a 420MB core file of that above process..any takers :)
>
> Ideas welcomed.
>
The best I have to offer at the moment is:
----
alex@berk:/usr/src$ strings core.4260 | sort | uniq -c | sort -r -n | head -n10
  77245         Acct-Delay-Time = 0
  72478         Realm = "NULL"
  69890         Acct-Authentic = RADIUS
  68526         Operator-Name = "1soas.ac.uk"
  67717         Service-Type = Framed-User
  67670         NAS-Port-Type = Ethernet
  67670         NAS-Identifier = "switches"
  67670         Huntgroup-Name = "cisco"
  55646         Acct-Status-Type = Interim-Update
  44341         Cisco-AVPair = "connect-progress=Call Up"
----

Well, it looks like something in the accounting side of things.

/me goes back to pondering

Cheers

--
Alexander Clouter
.sigmonster says: If we all work together, we can totally disrupt the system.

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
| Threaded
Open this post in threaded view
|

Re: freeradius memory leak

Alan DeKok-2
Alexander Clouter wrote:
> Well, it looks like something in the accounting side of things.
>
> /me goes back to pondering

  What accounting modules are you using?

  Alan DeKok.
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
| Threaded
Open this post in threaded view
|

Re: freeradius memory leak

Alexander Clouter
Alan DeKok <[hidden email]> wrote:
>
>> Well, it looks like something in the accounting side of things.
>>
>> /me goes back to pondering
>
> What accounting modules are you using?
>
All the interesting bits included below:
----
server lanwarden {
        ...

        preacct {
                preprocess
                suffix
        }

        accounting {
#               detail.lanwarden

                rewrite.quirk.wlc
                rewrite.called_station_id
                rewrite.calling_station_id
                client-info <--- just policy
                update request {
                        Operator-Name := "1%{config:local.MY.realm}"
                }

                # Reject Calling-Station-Id-less authentications
                if (!(Calling-Station-Id)) {
                        update reply {
                                Reply-Message := "No Calling-Station-Id"
                        }
                        reject
                }

                if (!(User-Name)) {
                        update reply {
                                Reply-Message := "No User-Name"
                        }
                        reject
                }

                validate_username <--- just policy

                # picked up by dot1x.decoupled-accounting
                detail.dot1x.acct

                attr_filter.eduroam-acct
        }

        ...
}

server eduroam {
        ...

        preacct {
                preprocess
                suffix
        }

        accounting {
#               detail.eduroam

                attr_filter.eduroam-acct

                rewrite.called_station_id
                rewrite.calling_station_id

                # Reject Calling-Station-Id-less authentications
                if (!(Calling-Station-Id)) {
                        update reply {
                                Reply-Message := "No Calling-Station-Id"
                        }
                        reject
                }

                if (!(User-Name)) {
                        update reply {
                                Reply-Message := "No User-Name"
                        }
                        reject
                }

                validate_username

                # Reject realm's authentications that are not for us early
                if (Realm != "%{config:local.MY.realm}") {
                        update reply {
                                Reply-Message := "Not @%{config:local.MY.realm} User-Name"
                        }
                        reject
                }

                # picked up by dot1x.decoupled-accounting
                detail.dot1x.acct
        }

        ...
}

# *very* lightly used, I doubt this would be the culprit
server login {
        ...

        preacct {
                preprocess

                suffix
        }

        accounting {
#               detail.login

                attr_filter.soas-acct

                if (!(User-Name)) {
                        update reply {
                                Reply-Message := "No User-Name"
                        }
                        reject
                }

                if (User-Name !~ /@/) {
                        update request {
                                User-Name := "%{User-Name}@%{config:local.MY.realm}"
                        }
                }

                # picked up by login.decoupled-accounting
                detail.login.acct
        }

        ...
}

server dot1x.decoupled-accounting {
        listen {
                type = detail
                filename = "${radacctdir}/journal/dot1x/detail.acct.*"
                load_factor = 10
        }

        preacct {
                preprocess

                soas_acct-unique

                suffix
        }

        accounting {
                # http://www.mail-archive.com/freeradius-users@.../msg54994.html
                sql.dot1x {
                        invalid = 2
                }
                if (invalid) {
                        ok
                }
                if (failed) {
                        # CISCO_ACCOUNTING_HACK fix "stop packet with zero session length. [user '[hidden email]', nas '192.0.2.1']"
                        if (Acct-Status-Type == Stop && (!(Acct-Session-Time) || Acct-Session-Time == 0) && Packet-Transmit-Counter > 5) {
                                ok
                        }
                }
        }
}

server login.decoupled-accounting {
        listen {
                type = detail
                filename = "${radacctdir}/journal/login/detail.acct.*"
                load_factor = 10
        }

        preacct {
                preprocess

                soas_acct-unique

                suffix
        }

        accounting {
                sql.login
        }
}
----
detail detail.login.acct {
        detailfile = ${radacctdir}/journal/login/detail.acct.%D:%H
        detailperm = 0600
        header = "%t"
        locking = yes
}
detail detail.dot1x.acct {
        detailfile = ${radacctdir}/journal/dot1x/detail.acct.%D:%H
        detailperm = 0600
        header = "%t"
        locking = yes
}
----

Let me know if you need anything else.

Cheers

--
Alexander Clouter
.sigmonster says: To be great is to be misunderstood.
                  -- Ralph Waldo Emerson

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html
| Threaded
Open this post in threaded view
|

Re: freeradius memory leak

Alan DeKok-2
Alexander Clouter wrote:
> All the interesting bits included below:

  Can you run the same config on an x86 box under valgrind?

> ----
> server lanwarden {

  OK... pretty much only rewrite && attr_filter are suspect.

> server eduroam {

  Same here.


> server dot1x.decoupled-accounting {

  SQL.

> server login.decoupled-accounting {
...
> Let me know if you need anything else.

  My guess is attr_filter.  There isn't much else that will make copies
of all of the accounting attributes.

  Unless you've uncovered another issue with the SQL module.  That code
is convoluted, and about 3x more complicated than it needs to be.

  Alan DeKok.
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/devel.html