I like the idea the Pieter is suggesting with communities to encode the route origin (country, continent, etc). I can imagine getting everyone to agree on and set communities could be tricky, but if the big networks that so generously provide transit (Internet2, GEANT, TransPAC, others?) were to adopt them that would cover off most of the use cases.

It's probably possible to achieve something similar by looking at each parties communities (if you can find where they're documented) and mixing in some as-path regular expressions, but this is a lot more work than a standard policy you could apply to all NREN peers using standard communities.

Thanks,

Dylan


On Thu, 22 Jul 2021 at 06:44, Eli Dart <dart@es.net> wrote:
Hi Pieter, all,

As you (and many others here) know, one of the difficulties in all of this is that routing metrics are not bandwidth-aware, latency-aware, or load-aware.  Some defaults try to incorporate bandwidth in a limited way, and some metrics can be explicitly configured to account for bandwidth and latency (in ESnet we do this with our IGP metrics), but those semantics are not dynamic.

When I was in the ESnet engineering group we used a combination of localpref and communities to manage things like this.  My understanding is that the architecture is still in place. The scheme is fairly detailed and complex, but once it's in place it provides a lot of power. Peer networks are grouped according to semantics (R&E, Cloud provider, commercial carrier, transit, etc), and preferred via localpref and BGP community based on peering capacity, whether they provide transit to other networks, etc. However, like in all things in this space, one has to be conscious of bandwidth, latency, etc. because the protocols don't do that on their own.  It's easy in a circumstance like this to prefer direct peering over transit (which is generally, but not always, our policy). It's also easy to customize policy at the peering and have the rest of the network follow the policy (local reasoning only).  But, it means doing a bunch of work up front.

Note that routing that appears weird may actually be correct depending on the needs of the people using the network. For R&E peering I would much rather have a clean 100G than a congested 10G, even if the 100G was 30 milliseconds longer. However, crossing the ocean to go down the road is probably (probably?) pathological. From my perspective, this is part of why the R&E networking community is a community - we work together to build and operate a global data fabric in the service of science. As we all make changes, sometimes we end up with pathological edge cases....this is why tools like NetSage are so valuable, and why community discussions such as this are important for all of us.

Thanks,

Eli


On Wed, Jul 21, 2021 at 8:39 AM Pieter de Boer <pieter.deboer@surf.nl> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

All,

I can totally second the usage of peeringDB which is the tool of preference for us (SURF) to get information a network (we we'll likely integrate this with our automation workflows). We also do publish some info in the RIPE database (https://apps.db.ripe.net/db-web-ui/query?searchtext=as-surfnet, which shows which ASes we provide transit for) or (https://apps.db.ripe.net/db-web-ui/query?bflag=false&dflag=false&rflag=true&searchtext=as1103&source=RIPE which shows with which networks we peer and what we send or accept from them, this object is uh not up to date....)

However neither of those two provide the information we need here, we need to make conscious decisions which network to prefer. Right now for the SURF ==> Kenia example we make the wrong decision. Below one of our three border routers (this box has direct peerings with Internet2/GEANT/Nordunet/AMS-IX/NL-IX/SpeedIX/Asteroid and transit, redundant or backup peers with Internet2/GEANT/Nordunet/exchanges/transit reside on the others)

In our peering policy we (SURF/AS1103) prefer to send traffic to customers directly, followed by research networks (local pref 300), private peers (local pref 160, metric 10), exchanges (local pref 160 metirc 20-60), transit (local pref 150/130). We prefer path via other research networks since in generall thay have higher capacity than via exchanges/transit.

In this example option 1, 2, 3, 4 fall in the same category research peers, with an equal AS-path length (3 AS’es). The battle is on between route 1, 2 and 3 (route 4 GEANT backup will loose from the GEANT primary due to set metrics). Since there is no direct tie breaker and we use JunOS (see https://www.juniper.net/documentation/en_US/junos/topics/reference/general/routing-protocols-address-representation.html) it is step 11, that does it so the route with the longest route-age wins for stability. The quick and dirty work-a-round flap both Internet2 peerings and we switch back to GEANT.
Route 5 is for us a pure exchange route (despite coming from Unbuntunet alliance for research and education networking), they have some presence in Amsterdam with interntional links and hence also connect to the AMS-IX. Route 6 and 7 fall in our transit category and are really last resort as they are over metered connections.

Of course as a human we can figure out a path via GEANT is probably far better than crossing the Atlantic ocean twice via Internet2. The question how do we get this information into our routers. I don’t see it to create manual exceptions in our routing policy for every weird route we find, this is something that will not scale. What I need is information that depicts the origin of this route something like Africa or maybe even a specific country, combined with the knowledge GEANT as a network is much closer to Africa than Internet2 it would make sense to prefer them.

My proposal if we could come to a common set of BGP communities we add to our routes that depict continents or countries we can then figure out with knowledge of the various networks which path between them is better. I would assume GEANT is able to offer us better routes to European, African and Asian research networks over Internet2. Where Internet2 probably has much better connectivity to north American and likely south American networks. If we could get geographic info in routing advertisements by communities (preferably a common set), we would know the route to Kenia via GEANT is better than Internet2.

Pieter

ps yesterdays talk was really interesting, nice visualization with Netsage and good discussions.

- --

SURFnet is transitioning towards SURF, my e-mail adres has changed accordingly  pieter.deboer@surfnet.nl got pieter.deboer@surf.nl

> -----Original Message-----
> From: Dale W. Carder <dwcarder@es.net>
> Sent: Tuesday, 20 July 2021 14:17
> To: routing-wg@lists.gna-g.net
> Subject: [Routing-wg] on publishing interconnection parameters
>
>
> On the call today the question was raised as to where one should
> publish, and correspondingly where one can find, route policy
> information for a given network.  I believe it was Warrick that
> mentioned the (historical) utility of traceroute.org as an example.
>
> At least for commercial peers here in the US, the use of peeringdb.com
> is probably the default interface used today for peering coordination.
> I would claim that it would be of significant utility to R&E networks
> for coordination as well.
>
> From there one can find relevant information such as:
> - URL to a Peering Policy web page w/ additional details
> - URL links to looking glasses (if applicable)
> - additional details in the notes field
>
> For the more specific case of bgp community support, at present this is
> still free-form text that has to go somewhere, but it's not always clear
> where to look.
>
> To pick on my friend Warrick, you can find a good example at:
> https://www.peeringdb.com/net/393
> Though perhaps in the notes field Warrick could say, "You can find
> supported bgp communities near the bottom of our magnificent 1,174
> line aut-num record in RADB"
>
> Or for us at https://www.peeringdb.com/net/940
> Perhaps we should add something to the tune of, "You can find our
> supported traffic engineering bgp communities at /dev/null, best of luck!
> But note for LHCONE VRF peers, we do support the mandatory knobs at
> https://twiki.cern.ch/twiki/bin/view/LHCONE/LhcOneVRF#BGP_communitie
> s"
>
> Dale
> _______________________________________________
> Routing-wg mailing list -- routing-wg@lists.gna-g.net
> To unsubscribe send an email to routing-wg-leave@lists.gna-g.net
-----BEGIN PGP SIGNATURE-----
Comment: Using gpg4o v6.0.124.9651 - https://www.gpg4o.com/
Charset: utf-8

iHUEAREIAB0WIQQmZT8oGCL/BcrPBy++sI1mNvdT0gUCYPg/owAKCRC+sI1mNvdT
0pm8AP9jHCF9q6Tn5yH1RBpsyrqH1YmuHdoPZLA421ZGDiBszwD+JNdoiznd+1ir
kwZ4rWGBK9rvj8U2V2+sqQDgy298nCE=
=XqGp
-----END PGP SIGNATURE-----_______________________________________________
Routing-wg mailing list -- routing-wg@lists.gna-g.net
To unsubscribe send an email to routing-wg-leave@lists.gna-g.net


--

Eli Dart, Network Engineer                          NOC: (510) 486-7600
ESnet Science Engagement Group                           (800) 333-7638
Lawrence Berkeley National Laboratory 
_______________________________________________
Routing-wg mailing list -- routing-wg@lists.gna-g.net
To unsubscribe send an email to routing-wg-leave@lists.gna-g.net