Hi Pieter, all,
As you (and many others here) know, one of the difficulties in all of this
is that routing metrics are not bandwidth-aware, latency-aware, or
load-aware. Some defaults try to incorporate bandwidth in a limited way,
and some metrics can be explicitly configured to account for bandwidth and
latency (in ESnet we do this with our IGP metrics), but those semantics are
not dynamic.
When I was in the ESnet engineering group we used a combination of
localpref and communities to manage things like this. My understanding is
that the architecture is still in place. The scheme is fairly detailed and
complex, but once it's in place it provides a lot of power. Peer networks
are grouped according to semantics (R&E, Cloud provider, commercial
carrier, transit, etc), and preferred via localpref and BGP community based
on peering capacity, whether they provide transit to other networks, etc.
However, like in all things in this space, one has to be conscious of
bandwidth, latency, etc. because the protocols don't do that on their own.
It's easy in a circumstance like this to prefer direct peering over transit
(which is generally, but not always, our policy). It's also easy to
customize policy at the peering and have the rest of the network follow the
policy (local reasoning only). But, it means doing a bunch of work up
front.
Note that routing that appears weird may actually be correct depending on
the needs of the people using the network. For R&E peering I would much
rather have a clean 100G than a congested 10G, even if the 100G was 30
milliseconds longer. However, crossing the ocean to go down the road is
probably (probably?) pathological. From my perspective, this is part of why
the R&E networking community is a community - we work together to build and
operate a global data fabric in the service of science. As we all make
changes, sometimes we end up with pathological edge cases....this is why
tools like NetSage are so valuable, and why community discussions such as
this are important for all of us.
Thanks,
Eli
On Wed, Jul 21, 2021 at 8:39 AM Pieter de Boer <pieter.deboer(a)surf.nl>
wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
All,
I can totally second the usage of peeringDB which is the tool of
preference for us (SURF) to get information a network (we we'll likely
integrate this with our automation workflows). We also do publish some info
in the RIPE database (
https://apps.db.ripe.net/db-web-ui/query?searchtext=as-surfnet, which
shows which ASes we provide transit for) or (
https://apps.db.ripe.net/db-web-ui/query?bflag=false&dflag=false&rf…
which shows with which networks we peer and what we send or accept from
them, this object is uh not up to date....)
However neither of those two provide the information we need here, we
need to make conscious decisions which network to prefer. Right now for the
SURF ==> Kenia example we make the wrong decision. Below one of our three
border routers (this box has direct peerings with
Internet2/GEANT/Nordunet/AMS-IX/NL-IX/SpeedIX/Asteroid and transit,
redundant or backup peers with Internet2/GEANT/Nordunet/exchanges/transit
reside on the others)
In our peering policy we (SURF/AS1103) prefer to send traffic to
customers directly, followed by research networks (local pref 300), private
peers (local pref 160, metric 10), exchanges (local pref 160 metirc 20-60),
transit (local pref 150/130). We prefer path via other research networks
since in generall thay have higher capacity than via exchanges/transit.
In this example option 1, 2, 3, 4 fall in the same category research
peers, with an equal AS-path length (3 AS’es). The battle is on between
route 1, 2 and 3 (route 4 GEANT backup will loose from the GEANT primary
due to set metrics). Since there is no direct tie breaker and we use JunOS
(see
https://www.juniper.net/documentation/en_US/junos/topics/reference/general/…)
it is step 11, that does it so the route with the longest route-age wins
for stability. The quick and dirty work-a-round flap both Internet2
peerings and we switch back to GEANT.
Route 5 is for us a pure exchange route (despite coming from Unbuntunet
alliance for research and education networking), they have some presence in
Amsterdam with interntional links and hence also connect to the AMS-IX.
Route 6 and 7 fall in our transit category and are really last resort as
they are over metered connections.
Of course as a human we can figure out a path via GEANT is probably far
better than crossing the Atlantic ocean twice via Internet2. The question
how do we get this information into our routers. I don’t see it to create
manual exceptions in our routing policy for every weird route we find, this
is something that will not scale. What I need is information that depicts
the origin of this route something like Africa or maybe even a specific
country, combined with the knowledge GEANT as a network is much closer to
Africa than Internet2 it would make sense to prefer them.
My proposal if we could come to a common set of BGP communities we add to
our routes that depict continents or countries we can then figure out with
knowledge of the various networks which path between them is better. I
would assume GEANT is able to offer us better routes to European, African
and Asian research networks over Internet2. Where Internet2 probably has
much better connectivity to north American and likely south American
networks. If we could get geographic info in routing advertisements by
communities (preferably a common set), we would know the route to Kenia via
GEANT is better than Internet2.
Pieter
ps yesterdays talk was really interesting, nice visualization with
Netsage and good discussions.
- --
SURFnet is transitioning towards SURF, my e-mail adres has changed
accordingly pieter.deboer(a)surfnet.nl got pieter.deboer(a)surf.nl
-----Original Message-----
From: Dale W. Carder <dwcarder(a)es.net>
Sent: Tuesday, 20 July 2021 14:17
To: routing-wg(a)lists.gna-g.net
Subject: [Routing-wg] on publishing interconnection parameters
On the call today the question was raised as to where one should
publish, and correspondingly where one can find, route policy
information for a given network. I believe it was Warrick that
mentioned the (historical) utility of
traceroute.org as an example.
At least for commercial peers here in the US, the use of
peeringdb.com
is probably the default interface used today for peering coordination.
I would claim that it would be of significant utility to R&E networks
for coordination as well.
From there one can find relevant information such as:
- URL to a Peering Policy web page w/ additional details
- URL links to looking glasses (if applicable)
- additional details in the notes field
For the more specific case of bgp community support, at present this is
still free-form text that has to go somewhere, but it's not always clear
where to look.
To pick on my friend Warrick, you can find a good example at:
https://www.peeringdb.com/net/393
Though perhaps in the notes field Warrick could say, "You can find
supported bgp communities near the bottom of our magnificent 1,174
line aut-num record in RADB"
Or for us at
https://www.peeringdb.com/net/940
Perhaps we should add something to the tune of, "You can find our
supported traffic engineering bgp communities at /dev/null, best of
luck!
But note for LHCONE VRF peers, we do support the
mandatory knobs at
https://twiki.cern.ch/twiki/bin/view/LHCONE/LhcOneVRF#BGP_communitie
s"
Dale
_______________________________________________
Routing-wg mailing list -- routing-wg(a)lists.gna-g.net
To unsubscribe send an email to routing-wg-leave(a)lists.gna-g.net
-----BEGIN PGP SIGNATURE-----
Comment: Using gpg4o v6.0.124.9651 -
https://www.gpg4o.com/
Charset: utf-8
iHUEAREIAB0WIQQmZT8oGCL/BcrPBy++sI1mNvdT0gUCYPg/owAKCRC+sI1mNvdT
0pm8AP9jHCF9q6Tn5yH1RBpsyrqH1YmuHdoPZLA421ZGDiBszwD+JNdoiznd+1ir
kwZ4rWGBK9rvj8U2V2+sqQDgy298nCE=
=XqGp
-----END PGP SIGNATURE-----_______________________________________________
Routing-wg mailing list -- routing-wg(a)lists.gna-g.net
To unsubscribe send an email to routing-wg-leave(a)lists.gna-g.net
--
Eli Dart, Network Engineer NOC: (510) 486-7600
ESnet Science Engagement Group (800) 333-7638
Lawrence Berkeley National Laboratory
_______________________________________________
Routing-wg mailing list -- routing-wg(a)lists.gna-g.net
To unsubscribe send an email to routing-wg-leave(a)lists.gna-g.net