Hi Routing Working Group,
On Wednesday the 28th of September (2 weeks from now) Nathaniel Mendoza and I will be presenting on Routing Security and why it matters at the CENIC conference in California.
I realize this is last minute, however, if you are going to be at the CENIC meeting and have experience deploying it on your networks, I would appreciate if you could create a couple slides and spend maybe 5 minutes discussing the deployment.
Of course hanging around for questions as well.
Thanks! Hans
Hi Hans, we won't be at the CENIC conference but I have a couple points that might be useful from our rollout of RPKI in REANNZ almost three years ago (Dec 2019).
At the time we rolled RPKI out we were something of an early adopter. Our greatest fear was we would be inundated with faults caused by routes being dropped because they failed the RPKI checks. From memory there were around 5000 bad routes in the full internet table at that time.
Our initial rollout was dropping invalid routes from our external neighbours, so this meant peering exchanges, NREN peers, and transit. We only recently started dropping invalids from customers. We applied the same policies to both commodity and NREN peers.
Over the first three months of our deployment we had only a couple faults caused by rejecting routes. Since then we've not had any faults.
The one take-away from this experience is the difficulties debugging a fault where we're rejecting a route. Unlike a route where we're taking a suboptimal path which you can see in your route table, a route that fails its ROA checks is simply missing. Working out where in your network you're rejecting it can be hard. On our Juniper MX platform this means walking around all the boxes with external peering sessions and checking the hidden routes. For our first fault we worked out another New Zealand based provider should have been announcing the missing route and accused them of not announcing it to REANNZ. Once we figured out it was us rejecting the route we had a chat with them about ROA's and the fault was quickly fixed.
So I think the lesson here is that your support staff need to be aware of RPKI and the challenges it can cause.
Thanks,
Dylan Hall Senior Network Engineer REANNZ Ltd
On Fri, 16 Sept 2022 at 04:17, Addleman, Hans C addlema@iu.edu wrote:
Hi Routing Working Group,
On Wednesday the 28th of September (2 weeks from now) Nathaniel Mendoza and I will be presenting on Routing Security and why it matters at the CENIC conference in California.
I realize this is last minute, however, if you are going to be at the CENIC meeting and have experience deploying it on your networks, I would appreciate if you could create a couple slides and spend maybe 5 minutes discussing the deployment.
Of course hanging around for questions as well.
Thanks!
Hans _______________________________________________ Routing-wg mailing list -- routing-wg@lists.gna-g.net To unsubscribe send an email to routing-wg-leave@lists.gna-g.net
Dylan, this is a great look at how you guys are dealing with RPKI! Thank you!
Would you mind if I turned your real world experience with the routes being hidden into a slide for my presentation? Full credit given of course 😊.
Thanks! Hans
From: Dylan Hall dylan.hall@reannz.co.nz Sent: Tuesday, September 20, 2022 2:39 AM To: Addleman, Hans C addlema@iu.edu Cc: routing-wg@lists.gna-g.net Subject: Re: [Routing-wg] Upcoming CENIC conference.
Hi Hans, we won't be at the CENIC conference but I have a couple points that might be useful from our rollout of RPKI in REANNZ almost three years ago (Dec 2019).
At the time we rolled RPKI out we were something of an early adopter. Our greatest fear was we would be inundated with faults caused by routes being dropped because they failed the RPKI checks. From memory there were around 5000 bad routes in the full internet table at that time.
Our initial rollout was dropping invalid routes from our external neighbours, so this meant peering exchanges, NREN peers, and transit. We only recently started dropping invalids from customers. We applied the same policies to both commodity and NREN peers.
Over the first three months of our deployment we had only a couple faults caused by rejecting routes. Since then we've not had any faults.
The one take-away from this experience is the difficulties debugging a fault where we're rejecting a route. Unlike a route where we're taking a suboptimal path which you can see in your route table, a route that fails its ROA checks is simply missing. Working out where in your network you're rejecting it can be hard. On our Juniper MX platform this means walking around all the boxes with external peering sessions and checking the hidden routes. For our first fault we worked out another New Zealand based provider should have been announcing the missing route and accused them of not announcing it to REANNZ. Once we figured out it was us rejecting the route we had a chat with them about ROA's and the fault was quickly fixed.
So I think the lesson here is that your support staff need to be aware of RPKI and the challenges it can cause.
Thanks,
Dylan Hall Senior Network Engineer REANNZ Ltd
On Fri, 16 Sept 2022 at 04:17, Addleman, Hans C <addlema@iu.edumailto:addlema@iu.edu> wrote: Hi Routing Working Group,
On Wednesday the 28th of September (2 weeks from now) Nathaniel Mendoza and I will be presenting on Routing Security and why it matters at the CENIC conference in California.
I realize this is last minute, however, if you are going to be at the CENIC meeting and have experience deploying it on your networks, I would appreciate if you could create a couple slides and spend maybe 5 minutes discussing the deployment.
Of course hanging around for questions as well.
Thanks! Hans _______________________________________________ Routing-wg mailing list -- routing-wg@lists.gna-g.netmailto:routing-wg@lists.gna-g.net To unsubscribe send an email to routing-wg-leave@lists.gna-g.netmailto:routing-wg-leave@lists.gna-g.net
Please do :)
Dylan
On Wed, 21 Sep 2022 at 4:33 AM, Addleman, Hans C addlema@iu.edu wrote:
Dylan, this is a great look at how you guys are dealing with RPKI! Thank you!
Would you mind if I turned your real world experience with the routes being hidden into a slide for my presentation? Full credit given of course 😊.
Thanks!
Hans
*From:* Dylan Hall dylan.hall@reannz.co.nz *Sent:* Tuesday, September 20, 2022 2:39 AM *To:* Addleman, Hans C addlema@iu.edu *Cc:* routing-wg@lists.gna-g.net *Subject:* Re: [Routing-wg] Upcoming CENIC conference.
Hi Hans, we won't be at the CENIC conference but I have a couple points that might be useful from our rollout of RPKI in REANNZ almost three years ago (Dec 2019).
At the time we rolled RPKI out we were something of an early adopter. Our greatest fear was we would be inundated with faults caused by routes being dropped because they failed the RPKI checks. From memory there were around 5000 bad routes in the full internet table at that time.
Our initial rollout was dropping invalid routes from our external neighbours, so this meant peering exchanges, NREN peers, and transit. We only recently started dropping invalids from customers. We applied the same policies to both commodity and NREN peers.
Over the first three months of our deployment we had only a couple faults caused by rejecting routes. Since then we've not had any faults.
The one take-away from this experience is the difficulties debugging a fault where we're rejecting a route. Unlike a route where we're taking a suboptimal path which you can see in your route table, a route that fails its ROA checks is simply missing. Working out where in your network you're rejecting it can be hard. On our Juniper MX platform this means walking around all the boxes with external peering sessions and checking the hidden routes. For our first fault we worked out another New Zealand based provider should have been announcing the missing route and accused them of not announcing it to REANNZ. Once we figured out it was us rejecting the route we had a chat with them about ROA's and the fault was quickly fixed.
So I think the lesson here is that your support staff need to be aware of RPKI and the challenges it can cause.
Thanks,
Dylan Hall
Senior Network Engineer
REANNZ Ltd
On Fri, 16 Sept 2022 at 04:17, Addleman, Hans C addlema@iu.edu wrote:
Hi Routing Working Group,
On Wednesday the 28th of September (2 weeks from now) Nathaniel Mendoza and I will be presenting on Routing Security and why it matters at the CENIC conference in California.
I realize this is last minute, however, if you are going to be at the CENIC meeting and have experience deploying it on your networks, I would appreciate if you could create a couple slides and spend maybe 5 minutes discussing the deployment.
Of course hanging around for questions as well.
Thanks!
Hans
Routing-wg mailing list -- routing-wg@lists.gna-g.net To unsubscribe send an email to routing-wg-leave@lists.gna-g.net
-
Addleman, Hans C -
Dylan Hall