Lists other than request in rlm_rest

classic Classic list List threaded Threaded
7 messages Options
| Threaded
Open this post in threaded view
|

Lists other than request in rlm_rest

Nathan Ward
Hi all,

I’d like to get control and reply attributes in to rlm_rest requests - we use control attributes for results of various bits of logic which we then translate in to reply attributes in post-auth if relevant - and it’d be useful to get those control attributes in to rest. Reply attributes in there would be useful as well, so we can “log" to a rest endpoint and include the request and reply (and control!) - a slightly different use case but same “tool”.

From what I can see, there doesn’t seem to be a way to have rlm_rest send attributes in the body (in JSON, let’s say) other than request attributes - the code I can see (rest_encode_json etc.) seems to be always looking at the request (request_t->packet->vps).

Is there a clever way I haven’t thought of to get control and reply attributes in to rest requests?

--
Nathan Ward


-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: Lists other than request in rlm_rest

Jorge Pereira-2
Hi Nathan,

Take a look at the option “data” as described in https://github.com/FreeRADIUS/freeradius-server/blob/v3.0.x/raddb/mods-available/rest#L78 <https://github.com/FreeRADIUS/freeradius-server/blob/v3.0.x/raddb/mods-available/rest#L78>

e.g:

post-auth {
   uri = “/path?foo=postauth-whatever”
   method = “post”
   data = “{ control_tmp_string_0 = \”%{control:Tmp-String-0}\” }”
   body = 'json'
   force_to = 'plain'
}

You mean something like that?
--
Jorge Pereira
[hidden email]




> Em 11 de jan de 2021, à(s) 18:21, Nathan Ward <[hidden email]> escreveu:
>
> Hi all,
>
> I’d like to get control and reply attributes in to rlm_rest requests - we use control attributes for results of various bits of logic which we then translate in to reply attributes in post-auth if relevant - and it’d be useful to get those control attributes in to rest. Reply attributes in there would be useful as well, so we can “log" to a rest endpoint and include the request and reply (and control!) - a slightly different use case but same “tool”.
>
> From what I can see, there doesn’t seem to be a way to have rlm_rest send attributes in the body (in JSON, let’s say) other than request attributes - the code I can see (rest_encode_json etc.) seems to be always looking at the request (request_t->packet->vps).
>
> Is there a clever way I haven’t thought of to get control and reply attributes in to rest requests?
>
> --
> Nathan Ward
>
>
> -
> List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: Lists other than request in rlm_rest

Nathan Ward
Hi Jorge,

> On 12/01/2021, at 11:43 AM, Jorge Pereira <[hidden email]> wrote:
>
> Hi Nathan,
>
> Take a look at the option “data” as described in https://github.com/FreeRADIUS/freeradius-server/blob/v3.0.x/raddb/mods-available/rest#L78 <https://github.com/FreeRADIUS/freeradius-server/blob/v3.0.x/raddb/mods-available/rest#L78>
>
> e.g:
>
> post-auth {
>   uri = “/path?foo=postauth-whatever”
>   method = “post”
>   data = “{ control_tmp_string_0 = \”%{control:Tmp-String-0}\” }”
>   body = 'json'
>   force_to = 'plain'
> }
>
> You mean something like that?

That sort of thing - though this requires that I know what all the attributes are.
Of course it’s my config, so I do, but I’d like to be able to just say “stick all the reply attributes in here” or something. Otherwise I have to have some logic to test for existence of each one, then include each instance of it, etc. etc.
If I was proxying I would not know what all the attributes are - in my use case I’m not proxying, but I could imagine it would be useful for others.

I’m not above doing a patch to rlm_rest to be able to include the request/reply/control attribute lists rather than packet->vps - assuming that that’s not a crazy bad idea for internal reasons I don’t understand.. Thoughts?

--
Nathan Ward


-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: Lists other than request in rlm_rest

Jorge Pereira-2
Assuming that you want to “reply” jSON packet. You could do something like:

1. Set the mods-enable/rest with something like:

>> post-auth {
>>  uri = “/path?foo=postauth-whatever”
>>  method = “post”
>>  data = "%{control:Tmp-String-0}"
>>  body = 'json'
>>  force_to = 'plain'
>> }

2. Just do your logic like:

If ("%{User-Name}" == “ tapioca”) {
   update control {
       &Tmp-String-0 := “{ \”myfield\”: \”hello tapioca\” }”
   }
} else {
   update control {
       &Tmp-String-0 := “{ \”error\”: \”bad things here\” }”
   }
}

--
Jorge Pereira
[hidden email]




> Em 11 de jan de 2021, à(s) 20:57, Nathan Ward <[hidden email]> escreveu:
>
> Hi Jorge,
>
>> On 12/01/2021, at 11:43 AM, Jorge Pereira <[hidden email]> wrote:
>>
>> Hi Nathan,
>>
>> Take a look at the option “data” as described in https://github.com/FreeRADIUS/freeradius-server/blob/v3.0.x/raddb/mods-available/rest#L78 <https://github.com/FreeRADIUS/freeradius-server/blob/v3.0.x/raddb/mods-available/rest#L78>
>>
>> e.g:
>>
>> post-auth {
>>  uri = “/path?foo=postauth-whatever”
>>  method = “post”
>>  data = “{ control_tmp_string_0 = \”%{control:Tmp-String-0}\” }”
>>  body = 'json'
>>  force_to = 'plain'
>> }
>>
>> You mean something like that?
>
> That sort of thing - though this requires that I know what all the attributes are.
> Of course it’s my config, so I do, but I’d like to be able to just say “stick all the reply attributes in here” or something. Otherwise I have to have some logic to test for existence of each one, then include each instance of it, etc. etc.
> If I was proxying I would not know what all the attributes are - in my use case I’m not proxying, but I could imagine it would be useful for others.
>
> I’m not above doing a patch to rlm_rest to be able to include the request/reply/control attribute lists rather than packet->vps - assuming that that’s not a crazy bad idea for internal reasons I don’t understand.. Thoughts?
>
> --
> Nathan Ward
>
>
> -
> List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: Lists other than request in rlm_rest

arr2036
In reply to this post by Nathan Ward

> That sort of thing - though this requires that I know what all the attributes are.
> Of course it’s my config, so I do, but I’d like to be able to just say “stick all the reply attributes in here” or something. Otherwise I have to have some logic to test for existence of each one, then include each instance of it, etc. etc.
> If I was proxying I would not know what all the attributes are - in my use case I’m not proxying, but I could imagine it would be useful for others.
>
> I’m not above doing a patch to rlm_rest to be able to include the request/reply/control attribute lists rather than packet->vps - assuming that that’s not a crazy bad idea for internal reasons I don’t understand.. Thoughts?

This is essentially a solved problem in v4 where we have the JSON module which can perform conversions on lists to JSON data.  That combined with the data parameter allows you to create arbitrarily formatted JSON data.

If you really wanted to add this to v3 I  guess the easiest way to would be to copy the contents of the control and reply lists to the request list.

update {
        request: += reply:[*]
        request: += control:[*]
}

There's unlikely to be overlap between the different sets of attributes.

Changing the JSON format in v3 to support lists wouldn't really work because it'd be a breaking change.  If you wanted to try out the current code in master branch and go the v4 route I'd be happy to address any issues that come up.

-Arran

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

signature.asc (849 bytes) Download Attachment
| Threaded
Open this post in threaded view
|

Re: Lists other than request in rlm_rest

Nathan Ward

> On 12/01/2021, at 11:04 PM, Arran Cudbard-Bell <[hidden email]> wrote:
>
>
>> That sort of thing - though this requires that I know what all the attributes are.
>> Of course it’s my config, so I do, but I’d like to be able to just say “stick all the reply attributes in here” or something. Otherwise I have to have some logic to test for existence of each one, then include each instance of it, etc. etc.
>> If I was proxying I would not know what all the attributes are - in my use case I’m not proxying, but I could imagine it would be useful for others.
>>
>> I’m not above doing a patch to rlm_rest to be able to include the request/reply/control attribute lists rather than packet->vps - assuming that that’s not a crazy bad idea for internal reasons I don’t understand.. Thoughts?
>
> This is essentially a solved problem in v4 where we have the JSON module which can perform conversions on lists to JSON data.  That combined with the data parameter allows you to create arbitrarily formatted JSON data.

Got it - good to hear there’s some tools coming to do this.
I’m also pondering python or perl or ruby or something to give me a tmp-string-x of JSON which I can stick in to the data param of a rest instance. We see about a thousand PPS at peak so should be tolerable performance..

> If you really wanted to add this to v3 I  guess the easiest way to would be to copy the contents of the control and reply lists to the request list.
>
> update {
> request: += reply:[*]
> request: += control:[*]
> }
>
> There's unlikely to be overlap between the different sets of attributes.

I guess if I’m doing this in post-auth right before I send the packet, it doesn’t matter if I have weird stuff in the request.. certainly a bit weird though.. Interesting idea, thanks!

I think there’s a variant here to avoid mashing attributes together, something like:

rest_request
update {
  request:[*] !* ANY
  request: += reply[*]
}
rest_reply
update {
  request:[*] !* ANY
  request: += control[*]
}
rest_ control

It means 3 HTTP requests, but means clear separation between which attributes are which.. I’ve got some things to test :-)

> Changing the JSON format in v3 to support lists wouldn't really work because it'd be a breaking change.  If you wanted to try out the current code in master branch and go the v4 route I'd be happy to address any issues that come up.

Ahh yep! Thanks for the offer - I think in my situation that’s going to be a bit of a hard sell, as that would mean V4 on front end RADIUS servers which if they break means customers call up - the usual concerns about beta code in production :-)
It would be possible if I only cared about accounting as I could do that on a backend server with a buffered relay thing - but that doesn’t help me with auth logging.

Having said that, I will give it a test to see if it works for my use case, so when v4 is released we can clean up whatever solution I come up with in the mean time.

--
Nathan Ward


-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
| Threaded
Open this post in threaded view
|

Re: Lists other than request in rlm_rest

arr2036


> On Jan 14, 2021, at 9:48 PM, Nathan Ward <[hidden email]> wrote:
>
>>
>> On 12/01/2021, at 11:04 PM, Arran Cudbard-Bell <[hidden email]> wrote:
>>
>>
>>> That sort of thing - though this requires that I know what all the attributes are.
>>> Of course it’s my config, so I do, but I’d like to be able to just say “stick all the reply attributes in here” or something. Otherwise I have to have some logic to test for existence of each one, then include each instance of it, etc. etc.
>>> If I was proxying I would not know what all the attributes are - in my use case I’m not proxying, but I could imagine it would be useful for others.
>>>
>>> I’m not above doing a patch to rlm_rest to be able to include the request/reply/control attribute lists rather than packet->vps - assuming that that’s not a crazy bad idea for internal reasons I don’t understand.. Thoughts?
>>
>> This is essentially a solved problem in v4 where we have the JSON module which can perform conversions on lists to JSON data.  That combined with the data parameter allows you to create arbitrarily formatted JSON data.
>
> Got it - good to hear there’s some tools coming to do this.
> I’m also pondering python or perl or ruby or something to give me a tmp-string-x of JSON which I can stick in to the data param of a rest instance. We see about a thousand PPS at peak so should be tolerable performance..
You might struggle to meet that with the current dynamic language modules.  Python is basically single threaded because of the GIL.  Python, Perl, Ruby etc all do complete conversions of the pair lists to/from language specific types.

I'm currently working on the python module in v4 to do just in time conversions between our boxed value format and Python's, which'll mean we don't waste CPU time converting values that'll never be used.  We may even be able to bypass the GIL and get everything running async, not sure yet.

>> If you really wanted to add this to v3 I  guess the easiest way to would be to copy the contents of the control and reply lists to the request list.
>>
>> update {
>> request: += reply:[*]
>> request: += control:[*]
>> }
>>
>> There's unlikely to be overlap between the different sets of attributes.
>
> I guess if I’m doing this in post-auth right before I send the packet, it doesn’t matter if I have weird stuff in the request.. certainly a bit weird though.. Interesting idea, thanks!
>
> I think there’s a variant here to avoid mashing attributes together, something like:
>
> rest_request
> update {
>  request:[*] !* ANY
>  request: += reply[*]
> }
> rest_reply
> update {
>  request:[*] !* ANY
>  request: += control[*]
> }
> rest_ control
>
> It means 3 HTTP requests, but means clear separation between which attributes are which.. I’ve got some things to test :-)
Yep that should work.

>> Changing the JSON format in v3 to support lists wouldn't really work because it'd be a breaking change.  If you wanted to try out the current code in master branch and go the v4 route I'd be happy to address any issues that come up.
>
> Ahh yep! Thanks for the offer - I think in my situation that’s going to be a bit of a hard sell, as that would mean V4 on front end RADIUS servers which if they break means customers call up - the usual concerns about beta code in production :-)
> It would be possible if I only cared about accounting as I could do that on a backend server with a buffered relay thing - but that doesn’t help me with auth logging.
>
> Having said that, I will give it a test to see if it works for my use case, so when v4 is released we can clean up whatever solution I come up with in the mean time.

OK.

-Arran


-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

signature.asc (849 bytes) Download Attachment