Hi,
If you have this realm block:
realm /@.+..+$/ { server dynamic server fallback.server.here accountingResponse on }
radsecproxy will start to send the request to fallback.server.here because the dynamic part didn't resolve yet: it's not blocking. Only as soon as the config for the dynamic realm is in place, when the dynamicLookupCommand had a result, it will continue with that host.
This results in part of the conversation going via one path, part of it via another. This breaks "the first" authentication for a realm.
Unless there is no fallback of course: or if the fallback is done in the lookup script (else at the end, which is what I'm using now). So, while this not being problematic for me ;-) I was wondering if someone else stumbled upon this, and whether we can have the dynamic lookup blocking for the request? That would allow fallback "as documented".
Regards, Paul
Hi Paul,
This results in part of the conversation going via one path, part of it via another.
I can’t notice this behaviour. But maybe I misunderstand you. What is your dynamic server block configuration? Which radsecproxy version are you running?
This breaks "the first" authentication for a realm.
Do you mean the first realm request is lost?
Regards, Ralf
Hi,
On 24/03/2021 06:10, Ralf Paffrath wrote:
Well this would be a simple example:
server etlr1.eduroam.org { type tls tls edupki }
server dynamic { type tls tls edupki dynamicLookupCommand /etc/radsecproxy/naptr-eduroam-lowercase.sh }
realm /@.+..+$/ { server dynamic server etlr1.eduroam.org accountingResponse on }
Which radsecproxy version are you running?
This happens with 1.8.1, 1.8.2 as well as 1.7.2. Did not test other versions.
This breaks "the first" authentication for a realm.
Do you mean the first realm request is lost?
If I start the server fresh, the first EAP request I do for a realm fails. The first packet is not blocking, but forwarded to the fallback server.
Regards, Paul
Hi,
Can you try "statusServer on” ?
server dynamic { type tls
statusServer on
What is the result?
Regards, Ralf
Hi Paul,
On 23.03.21, 13:04, "Paul Dekkers" paul.dekkers@surf.nl wrote: radsecproxy will start to send the request to fallback.server.here because the dynamic part didn't resolve yet: it's not blocking. Only as soon as the config for the dynamic realm is in place, when the dynamicLookupCommand had a result, it will continue with that host.
Just reading from the code:
The first request for a new realm will trigger the dynamic process. Essentially, a copy of the realm structure is created (for the actual realm to look up), including a copy of the dynamic server spec, but now including the realm to look up. This first request is implicitly placed in the queue for the dynamic server (before the lookupCommand has even been called). Now the server processes are started, which first calls the dynamicLookupCommand. During this time, the server is in 'startup' state, in which it will not be considered a valid server and will not get any more requests queued. If any more requests arrive for this realm, they will get sent to the other servers configured for the realm (if any). This is so that if the first request is retransmitted, as it took too long to resolve and connect to the dynamic server, it will fall back to the others. The clients timeout acting as an implicit timeout for the dynamic lookup (really, if it the lookup takes longer than the clients timeout we shouldn’t wait for it any longer)
This results in part of the conversation going via one path, part of it via another. This breaks "the first" authentication for a realm.
Of course there is the race condition if two clients happen to send their request within the time to resolve the dynamic server, one will get sent to the fallback server immediately and subsequent requests might get sent to the dynamic server later when it is established. From what I've seen in production, this case has been very rare (but maybe in larger countries its another story). However, even if the packets take different paths, this shouldn’t break the authentication - changing the paths due to lost requests (udp) or reset connections (tls) can happen any time.
Regards, Fabian
-- SWITCH Fabian Mauchle, Network Engineer Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland Phone +41 44 268 15 30, direct +41 44 268 15 39
Hi,
On 24/03/2021 08:49, Fabian Mauchle wrote:
It is really just one request. It's very easy to reproduce for me in that the first EAP authentication just always fails.
I actually think it's rare on servers with load, with frequent lookups for realms, but common on servers with little load.
Paul
Hi,
On 24.03.21, 08:49, "Fabian Mauchle" fabian.mauchle@switch.ch wrote: This first request is implicitly placed in the queue for the dynamic server (before the lookupCommand has even been called).
I have to corret myself here. Seems I've misunderstood that code block, and Pauls log (thanks!) clearly shows this. Reading it the right way, it looks quite intentional, not a bug.
The first request is _not_ placed on the dynamic servers queue. I read that one wrong, it only triggers the dynamic lookup, but the server selection is run again after the new realm structures have been set up. On this second (and any subsequent) passes, the dynamic server is ignored until its connection is established.
On 23.03.21, 13:04, "Paul Dekkers" paul.dekkers@surf.nl wrote: and whether we can have the dynamic lookup blocking for the request?
I will add it as a feature request. However, this might have quite some implications: if the lookup and connection setup is slow, this might block a realm for quite some time.
That would allow fallback "as documented".
Could you point me to that documentation? (I couldn’t spot it in the manpages; just so I can keep any documentation up to date).
Best regards, Fabian
Hi,
I had no Paul's log so I produced my own log.
And therefore it doesn’t break the first authentication. It is important to know that no request is lost and the first conversation is going via one fallback path and the next request later on via the dynamic path
But anyway we we will run into trouble. dynamic and fallback servers might include different realm blocks. So for dynamic realms which are not configured on the fallback servers the conversation might results in an Access-Reject for the first request and might results in an Access-Accept only in a second request via dynamic. This would break the service.
Best regards, Ralf
Hi,
On 25/03/2021 09:51, Ralf Paffrath wrote:
Hi,
I had no Paul's log so I produced my own log.
Happy to send it, Fabian asked it off-list but I'll forward,
In my case, it's a real that is both reachable via the fallback path and the primary path, yet the authentication fails.
Well it breaks for me.
I must admit that when Fabian asked for a pointer in the documentation, I don't really have one. I just more or less assumed this is how it worked, because that made sense to me. If you lookup realms dynamically, they may be only reachable via a dynamic path after all. So you're not sending something to fallback if the dynamic path is not resolved yet. And yes that is slow, but noone said dynamic lookups are fast - they aren't in Radiator either. That's why you need carefull tweaking of DNS settings too.
Anyway. I'll just use my workaround,
Regards, Paul
Hi,
thx. :-)
According to your logs your are right. I expanded my tests to EAP-TLS and voilà the conversation is ongoing via fallback (I tested different fallbacks) path and via primary path in the first authentication, but authentication never fails and no request is lost! To make it clear conversation starts with fallback path and is ongoing via dynamic path. I repeated the tests (stop the radsec etc.) and start again and again always the same picture. There is a successfully ongoing EAP-authentication via two path. Is that really surprising? If I’m not completely mistaken we have a proof of concept that radsecproxy is transparent to client and radius server. As long as you do not change the radius server during conversation auth should not fail. Could be the reason why authentication fails in your case.
Best regards, Ralf
-
Fabian Mauchle -
Paul Dekkers -
Ralf Paffrath