IP pools with rlm_sql_mongo + some general testing with MongoDB

classic Classic list List threaded Threaded
5 messages Options
| Threaded
Open this post in threaded view
|

IP pools with rlm_sql_mongo + some general testing with MongoDB

Benjamin Thompson
Firstly, many thanks to all concerned for creatiing a MongoDB module.

I have been doing a bit of testing of IP address allocation from pools
in MongoDB and I wanted to share some thoughts and notes (in no
particular order) in case they are useful:

1) I noticed that rlm_sql_mongo expects any returned data to be in a
field called "value". This means that for example for a document which
defines an IP address entry in the pool the actual IP must be stored
in the "value" field instead of something like "framed_ip_address":
{
    pool_key: "foo"
    pool_name: "bar"
    expiry_time: xxx
    value: "192.168.1.1"
}
I don't see any problem with this, but I mention it in case it is
useful to anyone else who tests the module.

2) MongoDB requests generally use a lot of curly brackets so when
callied via %{sql: xx} it is necessary to escape them with an extra
percent symbol. So in the actual request any "}" should be replaced
with "%}". This also maybe useful info to anyone testing for the first
time.

3) In my case I wanted to implement an IP pool with "sticky addresses"
based on a unique PPP username. One point to note is that the method
findAndModify() does not support sorting by an exact value. So for
example it cannot simulate something like "ORDER BY foo <> bar" which
is possible with other databases. In my case I decided to work around
this using two seperate requests: the first one does a lookup by exact
string to see if we can allocate the same IP, then if this does not
return anything do a second lookup and sort by expiry_time to find the
oldest free IP. Due to this I implemented all the database requests
directly in unlang directly in the authorize section instead of using
the sqlippool module.

4) I did at one point test the sqlippool module with a variation of
the above logic by calling one request in allocate_clear and another
in allocate_find. However for some reason I noticed that
allocate_clear was not geting called every time and somehow the server
was skipping it and calling allocate_find immediately. It could have
been a problem with my configuration so I will try to repocude the
problem if I get time.

5) In general I am very pleased with the performance. For a pool
database with 20000 entries on a modest 6 core virtual machine I
reached 6000 requests per second with radperf.

--
Ben Thompson
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: IP pools with rlm_sql_mongo + some general testing with MongoDB

Alan DeKok-2
On Feb 14, 2020, at 10:07 AM, Benjamin Thompson <[hidden email]> wrote:
>
> Firstly, many thanks to all concerned for creatiing a MongoDB module.

  That would largely be me.  :)

  The MongoC API is pretty good.  Mongo itself is rather more confusing.

> I have been doing a bit of testing of IP address allocation from pools
> in MongoDB and I wanted to share some thoughts and notes (in no
> particular order) in case they are useful:

  Yes, they are.

> 1) I noticed that rlm_sql_mongo expects any returned data to be in a
> field called "value". This means that for example for a document which
> defines an IP address entry in the pool the actual IP must be stored
> in the "value" field instead of something like "framed_ip_address":
> {
>    pool_key: "foo"
>    pool_name: "bar"
>    expiry_time: xxx
>    value: "192.168.1.1"
> }
> I don't see any problem with this, but I mention it in case it is
> useful to anyone else who tests the module.

  I'll make a note.

> 2) MongoDB requests generally use a lot of curly brackets so when
> callied via %{sql: xx} it is necessary to escape them with an extra
> percent symbol. So in the actual request any "}" should be replaced
> with "%}". This also maybe useful info to anyone testing for the first
> time.

  Sure.  I'll add some text around that,

> 3) In my case I wanted to implement an IP pool with "sticky addresses"
> based on a unique PPP username. One point to note is that the method
> findAndModify() does not support sorting by an exact value. So for
> example it cannot simulate something like "ORDER BY foo <> bar" which
> is possible with other databases. In my case I decided to work around
> this using two seperate requests: the first one does a lookup by exact
> string to see if we can allocate the same IP, then if this does not
> return anything do a second lookup and sort by expiry_time to find the
> oldest free IP. Due to this I implemented all the database requests
> directly in unlang directly in the authorize section instead of using
> the sqlippool module.

  Good!  FreeRADIUS can do just about anything. :)

> 4) I did at one point test the sqlippool module with a variation of
> the above logic by calling one request in allocate_clear and another
> in allocate_find. However for some reason I noticed that
> allocate_clear was not geting called every time and somehow the server
> was skipping it and calling allocate_find immediately. It could have
> been a problem with my configuration so I will try to repocude the
> problem if I get time.

  Hmm... IIRC allocate_clear isn't called every time?  I think it's once a second.

> 5) In general I am very pleased with the performance. For a pool
> database with 20000 entries on a modest 6 core virtual machine I
> reached 6000 requests per second with radperf.

  Very nice!

  The ISC "kea" DHCP server does 6K/s to it's in-memory store.  And less than 1K per second with databases.

  To me, that is a huge validation of the design of FreeRADIUS.  We write software that *isn't* crap.  And, we realize we're not smart enough to write our own database.  So we just use existing ones.

  The result is that our *untuned* performance to a database is as fast as the KEA DHCP *tuned* performance of an in-memory store.

  I think it's time to start pushing DHCP rather more aggressively. :)  We are faster than ISC, support more databases, have more configurable policies, etc.  The only thing we're missing is the ability to read the ISC config file format.

  Anyone want to back-port rlm_isc_dhcp from the "master" branch to v3. ? :)

  Alan DeKok.


-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: IP pools with rlm_sql_mongo + some general testing with MongoDB

Geaaru
Hi,
just my cent.
On Fri, 2020-02-14 at 10:54 -0500, Alan DeKok wrote:

> On Feb 14, 2020, at 10:07 AM, Benjamin Thompson <[hidden email]
> > wrote:
> > Firstly, many thanks to all concerned for creatiing a MongoDB
> > module.
>
>   That would largely be me.  :)
>   The MongoC API is pretty good.  Mongo itself is rather more
> confusing.
> > I have been doing a bit of testing of IP address allocation from
> > poolsin MongoDB and I wanted to share some thoughts and notes (in
> > noparticular order) in case they are useful:
>
>   Yes, they are.
> > 1) I noticed that rlm_sql_mongo expects any returned data to be in
> > afield called "value". This means that for example for a document
> > whichdefines an IP address entry in the pool the actual IP must be
> > storedin the "value" field instead of something like
> > "framed_ip_address":{   pool_key: "foo"   pool_name:
> > "bar"   expiry_time: xxx   value: "192.168.1.1"}I don't see any
> > problem with this, but I mention it in case it isuseful to anyone
> > else who tests the module.
>
>   I'll make a note.

The choice to use 'value' attribute is related to the fact that in
MongoDB doesn't exist a scalar response. It's always a document and
this is been a good compromise that could be used in a different use
case. This is better than implement a static response for sqlippool and
the driver is more simple.


> > 2) MongoDB requests generally use a lot of curly brackets so
> > whencallied via %{sql: xx} it is necessary to escape them with an
> > extrapercent symbol. So in the actual request any "}" should be
> > replacedwith "%}". This also maybe useful info to anyone testing
> > for the firsttime.
>
>   Sure.  I'll add some text around that,
> > 3) In my case I wanted to implement an IP pool with "sticky
> > addresses"based on a unique PPP username. One point to note is that
> > the methodfindAndModify() does not support sorting by an exact
> > value. So forexample it cannot simulate something like "ORDER BY
> > foo <> bar" whichis possible with other databases. In my case I
> > decided to work aroundthis using two seperate requests: the first
> > one does a lookup by exactstring to see if we can allocate the same
> > IP, then if this does notreturn anything do a second lookup and
> > sort by expiry_time to find theoldest free IP. Due to this I
> > implemented all the database requestsdirectly in unlang directly in
> > the authorize section instead of usingthe sqlippool module.
>
>   Good!  FreeRADIUS can do just about anything. :)

If you need to sort your data probably the better solution is to use
the aggregate and create a pipeline.

> > 4) I did at one point test the sqlippool module with a variation
> > ofthe above logic by calling one request in allocate_clear and
> > anotherin allocate_find. However for some reason I noticed
> > thatallocate_clear was not geting called every time and somehow the
> > serverwas skipping it and calling allocate_find immediately. It
> > could havebeen a problem with my configuration so I will try to
> > repocude theproblem if I get time.
>
>   Hmm... IIRC allocate_clear isn't called every time?  I think it's
> once a second.
> > 5) In general I am very pleased with the performance. For a
> > pooldatabase with 20000 entries on a modest 6 core virtual machine
> > Ireached 6000 requests per second with radperf.
>
>   Very nice!
>   The ISC "kea" DHCP server does 6K/s to it's in-memory store.  And
> less than 1K per second with databases.
>   To me, that is a huge validation of the design of FreeRADIUS.  We
> write software that *isn't* crap.  And, we realize we're not smart
> enough to write our own database.  So we just use existing ones.
>   The result is that our *untuned* performance to a database is as
> fast as the KEA DHCP *tuned* performance of an in-memory store.
>   I think it's time to start pushing DHCP rather more aggressively.
> :)  We are faster than ISC, support more databases, have more
> configurable policies, etc.  The only thing we're missing is the
> ability to read the ISC config file format.
>   Anyone want to back-port rlm_isc_dhcp from the "master" branch to
> v3. ? :)
>   Alan DeKok.
>
> -List info/subscribe/unsubscribe? See
> http://www.freeradius.org/list/users.html
>
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: IP pools with rlm_sql_mongo + some general testing with MongoDB

Benjamin Thompson
In reply to this post by Alan DeKok-2
On 14/02/2020, Alan DeKok <[hidden email]> wrote:
> On Feb 14, 2020, at 10:07 AM, Benjamin Thompson <[hidden email]>
> wrote:
>>
>> Firstly, many thanks to all concerned for creatiing a MongoDB module.
>
>   That would largely be me.  :)

Thanks Alan.



>> 3) In my case I wanted to implement an IP pool with "sticky addresses"
>> based on a unique PPP username. One point to note is that the method
>> findAndModify() does not support sorting by an exact value. So for
>> example it cannot simulate something like "ORDER BY foo <> bar" which
>> is possible with other databases. In my case I decided to work around
>> this using two seperate requests: the first one does a lookup by exact
>> string to see if we can allocate the same IP, then if this does not
>> return anything do a second lookup and sort by expiry_time to find the
>> oldest free IP. Due to this I implemented all the database requests
>> directly in unlang directly in the authorize section instead of using
>> the sqlippool module.
>
>   Good!  FreeRADIUS can do just about anything. :)

In case it is useful to anyone else, I ended up with something like this:

update reply {
    &Framed-IP-Address := "%{sql: db.radippool.findAndModify({'query':
{'$and': [{'pool_name': '%{control:Pool-Name}'%}, {'pool_key':
'%{User-Name}'%}]%}, 'update': {'$set': {'expiry_time': {'$date':
{'$numberLong': '%{expr: (%l + 3600) * 1000}'%}%}, 'nas_ip':
'%{NAS-IP-Address}', 'calling_station_id':
'%{Calling-Station-Id}'%}%}, 'fields': {'_id': 0, 'value': 1%}%})}"
}
if (&reply:Framed-IP-Address !~
/^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$/) {
    update reply {
        &Framed-IP-Address := "%{sql:
db.radippool.findAndModify({'query': {'$and': [{'pool_name':
'%{control:Pool-Name}'%}, {'expiry_time': {'$lt': {'$date':
{'$numberLong': '%{expr: %l * 1000}'%}%}%}%}]%}, 'update': {'$set':
{'expiry_time': {'$date': {'$numberLong': '%{expr: (%l + 3600) *
1000}'%}%}, 'pool_key': '%{User-Name}', 'nas_ip': '%{NAS-IP-Address}',
'calling_station_id': '%{Calling-Station-Id}'%}%}, 'fields': {'_id':
0, 'value': 1%}%})}"
    }
}


>
>> 4) I did at one point test the sqlippool module with a variation of
>> the above logic by calling one request in allocate_clear and another
>> in allocate_find. However for some reason I noticed that
>> allocate_clear was not geting called every time and somehow the server
>> was skipping it and calling allocate_find immediately. It could have
>> been a problem with my configuration so I will try to repocude the
>> problem if I get time.
>
>   Hmm... IIRC allocate_clear isn't called every time?  I think it's once a
> second.

Ahh that would explain it then. Thanks.


>   I think it's time to start pushing DHCP rather more aggressively. :)  We
> are faster than ISC, support more databases, have more configurable
> policies, etc.  The only thing we're missing is the ability to read the ISC
> config file format.


I do recommend FreeRADIUS to others (including for DHCP) and can't see
that anyone would go back to ISC after trying it. One thing though
which our clients are asking for more and more is DHCPv6  so on our
wishlist we are hoping that you will be get this in (v4) at some
point.

--
Ben Thompson
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: IP pools with rlm_sql_mongo + some general testing with MongoDB

Alan DeKok-2
On Feb 17, 2020, at 5:39 AM, Benjamin Thompson <[hidden email]> wrote:
> In case it is useful to anyone else, I ended up with something like this:

  Not trivial, but definitely powerful.

> I do recommend FreeRADIUS to others (including for DHCP) and can't see
> that anyone would go back to ISC after trying it.

  The things which are easy in ISC can be difficult in FreeRADIUS.  i.e. basic rules around static IPs, etc.

  The things which are impossible in FreeRADIUS are only slightly more difficult in FreeRADIUS.   FreeRADIUS has higher performance than ISC, is more flexible, supports more database back-ends, etc.

  In some sites we're seeing 8K DORAs per second with a Redis cluster back-end, and minimal logic.  When we add multiple round-trips to databases for various other policy checks, that drops to "only" 2K DORAs per second.

> One thing though
> which our clients are asking for more and more is DHCPv6  so on our
> wishlist we are hoping that you will be get this in (v4) at some
> point.

 Hmm...

$ cd ~/git/v4
$ ls src/protocols/dhcpv6
all.mk attrs.h base.c decode.c dhcpv6.h encode.c packet.c

$ ls src/tests/unit/protocols/dhcpv6
addresses.txt packet_domain-list.txt rfc6225.txt
bools.txt packet_ia-na.txt rfc6355.txt
dates.txt packet_ia-pd.txt rfc6939.txt
dictionary packet_ia-ta.txt rfc7078.txt
dns_wire_format.txt packet_ntp-server.txt rfc7600.txt
fixed_element_arrays.txt packet_sip-server-d.txt rfc8415.txt
integers.txt rfc3315.txt strings.txt
microsoft.txt rfc3319.txt tlvs.txt
packet.txt rfc3633.txt variable_element_arrays.txt
packet_AFTR-Name-rfc6334.txt rfc3646.txt
packet_client_server.txt rfc4704.txt

  I have no idea what that's about. :)

  The main difficulty with DHCPv6 is that it doesn't do TLVs.  It allocates all attributes from the same global attribute space.  Including sub-attributes.  Allowing this in v4 requires some fairly substantial rework of core pieces.  So that's taking time.

  Alan DeKok.


-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html