DHCP server crashed when receiving unknown option in Request

classic Classic list List threaded Threaded
7 messages Options
| Threaded
Open this post in threaded view
|

DHCP server crashed when receiving unknown option in Request

Chinnapaiyan, Nagamani
Hi,

We encountered a radiusd crash, when it received unknown option-145 in the Request packet(which is not defined in dictionary). Expected behavior is to ignore any unknown options and continue processing the packet.
Also please ensure(from code) radiusd is not crashing if the some known options are malformed in an incoming packet. Expected behavior is to ignore the packet and continue running.

Debug output: (seems crash is related to perl module)
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Opcode = Client-Message
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Hardware-Type = Ethernet
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Hardware-Address-Length = 6
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Hop-Count = 0
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Transaction-Id = 376356509
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Number-of-Seconds = 65535
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Flags = 0
...
...
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Hostname = "HP26D9AC"
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   Attr-145 = 0x01
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-V-I-Vendor-Class = 0x0000000b024850
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Parameter-Request-List = DHCP-Subnet-Mask
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Parameter-Request-List = DHCP-Classless-Static-Route
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Parameter-Request-List = DHCP-Static-Routes
...
...
...
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-Parameter-Request-List += $RAD_REQUEST{'DHCP-Parameter-Requ
est-List'} -> 'DHCP-Domain-Search'
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-Parameter-Request-List += $RAD_REQUEST{'DHCP-Parameter-Requ
est-List'} -> 'DHCP-Site-specific-28'
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-DHCP-Maximum-Msg-Size = $RAD_REQUEST{'DHCP-DHCP-Maximum-Msg
-Size'} -> '1500'
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-Server-IP-Address = $RAD_REQUEST{'DHCP-Server-IP-Address'}
-> '0.0.0.0'

(1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-V-I-Vendor-Class = $RAD_REQUEST{'DHCP-V-I-Vendor-Class'} -> ''

CONSISTENCY CHECK FAILED src/lib/util/pair.c[883]: VALUE_PAIR (raw/unknown) attribute 0x314a3a0 "Attr-145" data type incorrect.  Expected octets, got <INVALID>

ASSERT FAILED src/lib/util/pair.c[3050]: 0

CAUGHT SIGNAL: Aborted

Backtrace of last 18 frames:

/usr/lib64/freeradius/libfreeradius-util.so(fr_fault+0x1ae)[0x7f0aed10562e]

/usr/lib64/freeradius/libfreeradius-util.so(+0x148f9)[0x7f0aed1058f9]

/usr/lib64/freeradius/libfreeradius-util.so(fr_pair_verify+0x72d)[0x7f0aed11d60d]

/usr/lib64/freeradius/libfreeradius-util.so(fr_pair_add+0x27)[0x7f0aed11daf7]

/usr/lib64/freeradius/libfreeradius-util.so(fr_pair_make+0x148)[0x7f0aed11ea18]

/usr/lib64/freeradius/rlm_perl.so(+0x3169)[0x7f0ae3cb8169]

/usr/lib64/freeradius/rlm_perl.so(+0x33ec)[0x7f0ae3cb83ec]

/usr/lib64/freeradius/rlm_perl.so(+0x56f4)[0x7f0ae3cba6f4]

/usr/lib64/freeradius/libfreeradius-unlang.so(+0x118bc)[0x7f0aed58f8bc]

/usr/lib64/freeradius/libfreeradius-unlang.so(unlang_interpret+0x378)[0x7f0aed58c508]

/usr/lib64/freeradius/proto_dhcpv4_process.so(+0x156d)[0x7f0ae700e56d]

/usr/lib64/freeradius/libfreeradius-io.so(+0x14a83)[0x7f0aed36da83]

/usr/lib64/freeradius/libfreeradius-util.so(fr_event_service+0x23b)[0x7f0aed112bdb]

/usr/lib64/freeradius/libfreeradius-util.so(fr_event_loop+0x20)[0x7f0aed112fa0]

/usr/lib64/freeradius/libfreeradius-server.so(main_loop_start+0x4e)[0x7f0aed7e466e]

/usr/sbin/radiusd(main+0xe14)[0x404b64]

/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f0aeb123555]

/usr/sbin/radiusd[0x405021]

No panic action set

After we added 145 in the dictionary, it started processing the request packet successfully.

Regards,
Nagamani Chinnapaiyan

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

RE: DHCP server crashed when receiving unknown option in Request

Chinnapaiyan, Nagamani
We are using freeradius 4.0.x version(master branch).

Tue Jun 30 06:18:57 2020: Info  : FreeRADIUS Version 4.0.0
Tue Jun 30 06:18:57 2020: Info  : Copyright 1999-2019 The FreeRADIUS server project and contributors
Tue Jun 30 06:18:57 2020: Info  : There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
Tue Jun 30 06:18:57 2020: Info  : PARTICULAR PURPOSE
Tue Jun 30 06:18:57 2020: Info  : You may redistribute copies of FreeRADIUS under the terms of the
Tue Jun 30 06:18:57 2020: Info  : GNU General Public License
Tue Jun 30 06:18:57 2020: Info  : For more information about these matters, see the file named COPYRIGHT
Tue Jun 30 06:18:58 2020: Info  : Starting - reading configuration files ...
Tue Jun 30 06:18:58 2020: Debug : Including dictionary file "/etc/raddb/dictionary"
Tue Jun 30 06:18:58 2020: Debug : including configuration file /etc/raddb/radiusd.conf
Tue Jun 30 06:18:58 2020: Debug : Including files in directory "/etc/raddb/template.d/"
Tue Jun 30 06:18:58 2020: Debug : including configuration file /etc/raddb/template.d/default

Regards,
Nagamani Chinnapaiyan

From: Chinnapaiyan, Nagamani
Sent: Tuesday, June 30, 2020 1:18 PM
To: '[hidden email]' <[hidden email]>
Subject: DHCP server crashed when receiving unknown option in Request

Hi,

We encountered a radiusd crash, when it received unknown option-145 in the Request packet(which is not defined in dictionary). Expected behavior is to ignore any unknown options and continue processing the packet.
Also please ensure(from code) radiusd is not crashing if the some known options are malformed in an incoming packet. Expected behavior is to ignore the packet and continue running.

Debug output: (seems crash is related to perl module)
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Opcode = Client-Message
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Hardware-Type = Ethernet
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Hardware-Address-Length = 6
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Hop-Count = 0
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Transaction-Id = 376356509
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Number-of-Seconds = 65535
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Flags = 0
...
...
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Hostname = "HP26D9AC"
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   Attr-145 = 0x01
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-V-I-Vendor-Class = 0x0000000b024850
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Parameter-Request-List = DHCP-Subnet-Mask
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Parameter-Request-List = DHCP-Classless-Static-Route
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   DHCP-Parameter-Request-List = DHCP-Static-Routes
...
...
...
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-Parameter-Request-List += $RAD_REQUEST{'DHCP-Parameter-Requ
est-List'} -> 'DHCP-Domain-Search'
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-Parameter-Request-List += $RAD_REQUEST{'DHCP-Parameter-Requ
est-List'} -> 'DHCP-Site-specific-28'
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-DHCP-Maximum-Msg-Size = $RAD_REQUEST{'DHCP-DHCP-Maximum-Msg
-Size'} -> '1500'
(1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-Server-IP-Address = $RAD_REQUEST{'DHCP-Server-IP-Address'}
-> '0.0.0.0'

(1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-V-I-Vendor-Class = $RAD_REQUEST{'DHCP-V-I-Vendor-Class'} -> ''

CONSISTENCY CHECK FAILED src/lib/util/pair.c[883]: VALUE_PAIR (raw/unknown) attribute 0x314a3a0 "Attr-145" data type incorrect.  Expected octets, got <INVALID>

ASSERT FAILED src/lib/util/pair.c[3050]: 0

CAUGHT SIGNAL: Aborted

Backtrace of last 18 frames:

/usr/lib64/freeradius/libfreeradius-util.so(fr_fault+0x1ae)[0x7f0aed10562e]

/usr/lib64/freeradius/libfreeradius-util.so(+0x148f9)[0x7f0aed1058f9]

/usr/lib64/freeradius/libfreeradius-util.so(fr_pair_verify+0x72d)[0x7f0aed11d60d]

/usr/lib64/freeradius/libfreeradius-util.so(fr_pair_add+0x27)[0x7f0aed11daf7]

/usr/lib64/freeradius/libfreeradius-util.so(fr_pair_make+0x148)[0x7f0aed11ea18]

/usr/lib64/freeradius/rlm_perl.so(+0x3169)[0x7f0ae3cb8169]

/usr/lib64/freeradius/rlm_perl.so(+0x33ec)[0x7f0ae3cb83ec]

/usr/lib64/freeradius/rlm_perl.so(+0x56f4)[0x7f0ae3cba6f4]

/usr/lib64/freeradius/libfreeradius-unlang.so(+0x118bc)[0x7f0aed58f8bc]

/usr/lib64/freeradius/libfreeradius-unlang.so(unlang_interpret+0x378)[0x7f0aed58c508]

/usr/lib64/freeradius/proto_dhcpv4_process.so(+0x156d)[0x7f0ae700e56d]

/usr/lib64/freeradius/libfreeradius-io.so(+0x14a83)[0x7f0aed36da83]

/usr/lib64/freeradius/libfreeradius-util.so(fr_event_service+0x23b)[0x7f0aed112bdb]

/usr/lib64/freeradius/libfreeradius-util.so(fr_event_loop+0x20)[0x7f0aed112fa0]

/usr/lib64/freeradius/libfreeradius-server.so(main_loop_start+0x4e)[0x7f0aed7e466e]

/usr/sbin/radiusd(main+0xe14)[0x404b64]

/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f0aeb123555]

/usr/sbin/radiusd[0x405021]

No panic action set

After we added 145 in the dictionary, it started processing the request packet successfully.

Regards,
Nagamani Chinnapaiyan

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: DHCP server crashed when receiving unknown option in Request

Alan DeKok-2
In reply to this post by Chinnapaiyan, Nagamani
On Jun 30, 2020, at 3:48 AM, Chinnapaiyan, Nagamani <[hidden email]> wrote:
>
> Hi,
>
> We encountered a radiusd crash, when it received unknown option-145 in the Request packet(which is not defined in dictionary). Expected behavior is to ignore any unknown options and continue processing the packet.
> Also please ensure(from code) radiusd is not crashing if the some known options are malformed in an incoming packet. Expected behavior is to ignore the packet and continue running.

  We do know that.  We have literally hundreds of tests for this, and other circumstances.

> (1470)  Tue Jun 30 06:21:11 2020 : Debug :   perl - &request:DHCP-V-I-Vendor-Class = $RAD_REQUEST{'DHCP-V-I-Vendor-Class'} -> ''
>
> CONSISTENCY CHECK FAILED src/lib/util/pair.c[883]: VALUE_PAIR (raw/unknown) attribute 0x314a3a0 "Attr-145" data type incorrect.  Expected octets, got <INVALID>
>
> ASSERT FAILED src/lib/util/pair.c[3050]: 0

  The current "master" branch has no such messages at line 883 or 3050.  Please ensure that you're running the latest branch.  The code in "master" changes a lot.  So it's not productive for us to track down issues which may have already been fixed months ago.

  Alan DeKok.


-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

RE: DHCP server crashed when receiving unknown option in Request

Chinnapaiyan, Nagamani
In reply to this post by Chinnapaiyan, Nagamani
> We do know that.  We have literally hundreds of tests for this, and other circumstances.
You mean you already know it will crash if unknow options are incoming? And you have testcases which are failing because of this?


>  The current "master" branch has no such messages at line 883 or 3050.  Please ensure that you're running the latest branch.  The code in "master" changes a lot.  So it's not productive for us to track down issues which may have already been fixed months ago.
We will checkout latest master branch and try. But the current one is taken recently within a month. We will check the latest commit anyway and get back.

Regards,
Nagamani Chinnapaiyan
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: DHCP server crashed when receiving unknown option in Request

Alan DeKok-2
On Jun 30, 2020, at 10:21 AM, Chinnapaiyan, Nagamani <[hidden email]> wrote:
>
>> We do know that.  We have literally hundreds of tests for this, and other circumstances.
> You mean you already know it will crash if unknow options are incoming? And you have testcases which are failing because of this?

   I mean that you should pay attention to how your comments come across.  They are borderline rude.

  Your comment was "please ensure(from code) radiusd is not crashing"  My answer is "yes, we know already that the server isn't supposed to crash".

  If the server crashes, it's a bug, and we'll fix it.  We have been _very_ clear on that subject for the past 20 years.  We've said so on this list many times.  We do _not_ need someone to tell us this.

  We have hundreds of tests to ensure that the server  *doesn't* crash when it gets unknown attributes.  We have hundreds of tests to ensure that the server deals with all kinds of bizarre corner cases.  If we missed a case, we will add a test.

  If you find a crash, you can open a bug report.  You should include enough information that we can easily reproduce the issue.  We can then add tests to be sure that the issue doesn't come back.

>> The current "master" branch has no such messages at line 883 or 3050.  Please ensure that you're running the latest branch.  The code in "master" changes a lot.  So it's not productive for us to track down issues which may have already been fixed months ago.
> We will checkout latest master branch and try. But the current one is taken recently within a month. We will check the latest commit anyway and get back.

  That should help.

  Alan DeKok.


-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

DHCP server crashed when receiving unknown option in Request

Chinnapaiyan, Nagamani
In reply to this post by Chinnapaiyan, Nagamani
>   I mean that you should pay attention to how your comments come across.  They are borderline rude.
>
>  Your comment was "please ensure(from code) radiusd is not crashing"  My answer is "yes, we know already that the server isn't supposed to crash".

Really sorry about that.

Regards,
Nagamani Chinnapaiyan

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: DHCP server crashed when receiving unknown option in Request

Alan DeKok-2
  I've pushed a fix to the "master" branch, and added a test so that this doesn't happen again.

  Alan DeKok.


-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html