Hi Pieter, all,
As you (and many others here) know, one of the difficulties in all of this
is that routing metrics are not bandwidth-aware, latency-aware, or
load-aware. Some defaults try to incorporate bandwidth in a limited way,
and some metrics can be explicitly configured to account for bandwidth and
latency (in ESnet we do this with our IGP metrics), but those semantics are
not dynamic.
When I was in the ESnet engineering group we used a combination of
localpref and communities to manage things like this. My understanding is
that the architecture is still in place. The scheme is fairly detailed and
complex, but once it's in place it provides a lot of power. Peer networks
are grouped according to semantics (R&E, Cloud provider, commercial
carrier, transit, etc), and preferred via localpref and BGP community based
on peering capacity, whether they provide transit to other networks, etc.
However, like in all things in this space, one has to be conscious of
bandwidth, latency, etc. because the protocols don't do that on their own.
It's easy in a circumstance like this to prefer direct peering over transit
(which is generally, but not always, our policy). It's also easy to
customize policy at the peering and have the rest of the network follow the
policy (local reasoning only). But, it means doing a bunch of work up
front.
Note that routing that appears weird may actually be correct depending on
the needs of the people using the network. For R&E peering I would much
rather have a clean 100G than a congested 10G, even if the 100G was 30
milliseconds longer. However, crossing the ocean to go down the road is
probably (probably?) pathological. From my perspective, this is part of why
the R&E networking community is a community - we work together to build and
operate a global data fabric in the service of science. As we all make
changes, sometimes we end up with pathological edge cases....this is why
tools like NetSage are so valuable, and why community discussions such as
this are important for all of us.
Thanks,
Eli
On Wed, Jul 21, 2021 at 8:39 AM Pieter de Boer <pieter.deboer(a)surf.nl>
wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> All,
>
> I can totally second the usage of peeringDB which is the tool of
> preference for us (SURF) to get information a network (we we'll likely
> integrate this with our automation workflows). We also do publish some info
> in the RIPE database (
> https://apps.db.ripe.net/db-web-ui/query?searchtext=as-surfnet, which
> shows which ASes we provide transit for) or (
> https://apps.db.ripe.net/db-web-ui/query?bflag=false&dflag=false&rflag=true…
> which shows with which networks we peer and what we send or accept from
> them, this object is uh not up to date....)
>
> However neither of those two provide the information we need here, we need
> to make conscious decisions which network to prefer. Right now for the SURF
> ==> Kenia example we make the wrong decision. Below one of our three border
> routers (this box has direct peerings with
> Internet2/GEANT/Nordunet/AMS-IX/NL-IX/SpeedIX/Asteroid and transit,
> redundant or backup peers with Internet2/GEANT/Nordunet/exchanges/transit
> reside on the others)
>
> In our peering policy we (SURF/AS1103) prefer to send traffic to customers
> directly, followed by research networks (local pref 300), private peers
> (local pref 160, metric 10), exchanges (local pref 160 metirc 20-60),
> transit (local pref 150/130). We prefer path via other research networks
> since in generall thay have higher capacity than via exchanges/transit.
>
> In this example option 1, 2, 3, 4 fall in the same category research
> peers, with an equal AS-path length (3 AS’es). The battle is on between
> route 1, 2 and 3 (route 4 GEANT backup will loose from the GEANT primary
> due to set metrics). Since there is no direct tie breaker and we use JunOS
> (see
> https://www.juniper.net/documentation/en_US/junos/topics/reference/general/…)
> it is step 11, that does it so the route with the longest route-age wins
> for stability. The quick and dirty work-a-round flap both Internet2
> peerings and we switch back to GEANT.
> Route 5 is for us a pure exchange route (despite coming from Unbuntunet
> alliance for research and education networking), they have some presence in
> Amsterdam with interntional links and hence also connect to the AMS-IX.
> Route 6 and 7 fall in our transit category and are really last resort as
> they are over metered connections.
>
> Of course as a human we can figure out a path via GEANT is probably far
> better than crossing the Atlantic ocean twice via Internet2. The question
> how do we get this information into our routers. I don’t see it to create
> manual exceptions in our routing policy for every weird route we find, this
> is something that will not scale. What I need is information that depicts
> the origin of this route something like Africa or maybe even a specific
> country, combined with the knowledge GEANT as a network is much closer to
> Africa than Internet2 it would make sense to prefer them.
>
> My proposal if we could come to a common set of BGP communities we add to
> our routes that depict continents or countries we can then figure out with
> knowledge of the various networks which path between them is better. I
> would assume GEANT is able to offer us better routes to European, African
> and Asian research networks over Internet2. Where Internet2 probably has
> much better connectivity to north American and likely south American
> networks. If we could get geographic info in routing advertisements by
> communities (preferably a common set), we would know the route to Kenia via
> GEANT is better than Internet2.
>
> Pieter
>
> ps yesterdays talk was really interesting, nice visualization with Netsage
> and good discussions.
>
> - --
>
> SURFnet is transitioning towards SURF, my e-mail adres has changed
> accordingly pieter.deboer(a)surfnet.nl got pieter.deboer(a)surf.nl
>
> > -----Original Message-----
> > From: Dale W. Carder <dwcarder(a)es.net>
> > Sent: Tuesday, 20 July 2021 14:17
> > To: routing-wg(a)lists.gna-g.net
> > Subject: [Routing-wg] on publishing interconnection parameters
> >
> >
> > On the call today the question was raised as to where one should
> > publish, and correspondingly where one can find, route policy
> > information for a given network. I believe it was Warrick that
> > mentioned the (historical) utility of traceroute.org as an example.
> >
> > At least for commercial peers here in the US, the use of peeringdb.com
> > is probably the default interface used today for peering coordination.
> > I would claim that it would be of significant utility to R&E networks
> > for coordination as well.
> >
> > From there one can find relevant information such as:
> > - URL to a Peering Policy web page w/ additional details
> > - URL links to looking glasses (if applicable)
> > - additional details in the notes field
> >
> > For the more specific case of bgp community support, at present this is
> > still free-form text that has to go somewhere, but it's not always clear
> > where to look.
> >
> > To pick on my friend Warrick, you can find a good example at:
> > https://www.peeringdb.com/net/393
> > Though perhaps in the notes field Warrick could say, "You can find
> > supported bgp communities near the bottom of our magnificent 1,174
> > line aut-num record in RADB"
> >
> > Or for us at https://www.peeringdb.com/net/940
> > Perhaps we should add something to the tune of, "You can find our
> > supported traffic engineering bgp communities at /dev/null, best of luck!
> > But note for LHCONE VRF peers, we do support the mandatory knobs at
> > https://twiki.cern.ch/twiki/bin/view/LHCONE/LhcOneVRF#BGP_communitie
> > s"
> >
> > Dale
> > _______________________________________________
> > Routing-wg mailing list -- routing-wg(a)lists.gna-g.net
> > To unsubscribe send an email to routing-wg-leave(a)lists.gna-g.net
> -----BEGIN PGP SIGNATURE-----
> Comment: Using gpg4o v6.0.124.9651 - https://www.gpg4o.com/
> Charset: utf-8
>
> iHUEAREIAB0WIQQmZT8oGCL/BcrPBy++sI1mNvdT0gUCYPg/owAKCRC+sI1mNvdT
> 0pm8AP9jHCF9q6Tn5yH1RBpsyrqH1YmuHdoPZLA421ZGDiBszwD+JNdoiznd+1ir
> kwZ4rWGBK9rvj8U2V2+sqQDgy298nCE=
> =XqGp
> -----END PGP SIGNATURE-----_______________________________________________
> Routing-wg mailing list -- routing-wg(a)lists.gna-g.net
> To unsubscribe send an email to routing-wg-leave(a)lists.gna-g.net
>
--
Eli Dart, Network Engineer NOC: (510) 486-7600
ESnet Science Engagement Group (800) 333-7638
Lawrence Berkeley National Laboratory
On the call today the question was raised as to where one should
publish, and correspondingly where one can find, route policy
information for a given network. I believe it was Warrick that
mentioned the (historical) utility of traceroute.org as an example.
At least for commercial peers here in the US, the use of peeringdb.com
is probably the default interface used today for peering coordination.
I would claim that it would be of significant utility to R&E networks
for coordination as well.
From there one can find relevant information such as:
- URL to a Peering Policy web page w/ additional details
- URL links to looking glasses (if applicable)
- additional details in the notes field
For the more specific case of bgp community support, at present this is
still free-form text that has to go somewhere, but it's not always clear
where to look.
To pick on my friend Warrick, you can find a good example at:
https://www.peeringdb.com/net/393
Though perhaps in the notes field Warrick could say, "You can find
supported bgp communities near the bottom of our magnificent 1,174
line aut-num record in RADB"
Or for us at https://www.peeringdb.com/net/940
Perhaps we should add something to the tune of, "You can find our
supported traffic engineering bgp communities at /dev/null, best of luck!
But note for LHCONE VRF peers, we do support the mandatory knobs at
https://twiki.cern.ch/twiki/bin/view/LHCONE/LhcOneVRF#BGP_communities"
Dale
Hello Routing-WG members
Thanks everyone who came to the meeting this morning.
Our next meetings are:
APAN Routing Working Group BOF, 9am August 3 Indonesia local time (10PM August 2, US Eastern time)
4 weeks from today, same time, walking through the examples (see notes below), 6am August 17, US Eastern time
MANRS Talk by Steve Wallace, 8pm September 7, US Eastern time
We meant to record today’s talk but that didn’t happen unfortunately. If there’s enough interest/need we can try to do another in the opposite time zone. The slides are available online here:
https://docs.google.com/presentation/d/1x07cL8-o38sWHZd-RXSnZLinkj0t02IBXjp…
We walked through 4 examples, although there’s a 5th in the slides (slides 17 and 18) for very strange routes between Singapore and New Zealand. These included unexpected the following unexpected routes:
NIA (S. Korea) to Microsoft (Singapore)
Nanyang Technological University (Singapore) to Education Bureau, Kaohsiung City Government (Taiwan)
CERN (Geneva) to KISTI (S. Korea)
KENET to SURFnet
National University of Singapore to University Otago (New Zealand)
We’d like to put together teams to start to address these, with the hopes of using these examples to talk about what infrastructure/documentation we need to address these more generally, and reviewing them at the mid August meeting. Please respond to any of the chairs if you want to be involved in any of these. If the co-chairs decide you’re needed, you may get looped in even if you don’t tell us. We’re hoping these teams will start working on these in the next week with the hopes of having something to report in 4 weeks. Likely these will also be discussed on the Slack channel – so don’t forget to watch there as well.
Please get in touch with any questions or comments – see you in 2 weeks at APAN and then at our next regular meeting 2 weeks after that.
-jennifer, on behalf of the Routing WG co-chairs
------------------------------------------------
Dr. Jennifer M. Schopf
Director, International Networks
Director, Engagement and Performance Operations Center
Indiana University
The slides we’ll be using are here, which everyone should have access to
https://docs.google.com/presentation/d/1x07cL8-o38sWHZd-RXSnZLinkj0t02IBXjp…
But mostly we’ll be walking through the live NetSage pages listed in them.
-j
------------------------------------------------
Dr. Jennifer M. Schopf
Director, International Networks
Director, Engagement and Performance Operations Center
Indiana University
From: "Addleman, Hans C" <addlema(a)iu.edu>
Date: Monday, July 19, 2021 at 2:00 PM
To: "routing-wg(a)lists.gna-g.net" <routing-wg(a)lists.gna-g.net>
Subject: [Routing-wg] Routing Working Group Meeting 3 - Tool Talk - NetSage Reminder
Greetings Routing Working Group,
Reminder that we will have our next meeting tomorrow July 20 at 6:00am US Eastern time. This will be the next in our series of Tool Talks. Below please find the abstract for the talk on NetSage. You should have a calendar invite, however, I have attached an invite to this mail as well just in case.
Title- NetSage as a tool to find routing anomalies
Speaker – Jennifer Schopf
Abstract
The NetSage measurement and monitoring system (http://portal.netsage.global) is most often used by network owners to understand what data is using the circuits, what performance data transfers are experiencing, and for identifying changes of behavior over a set of links. However, it can also be used to identify end-to-end transfers that may not be taking the shortest path, especially in international settings.
This talk will give a brief overview of NetSage and its current deployments, then will show some real life examples of unusual routes – which may or may not be the preferred paths for data.
-----------------------------------------------
Hans Addleman is inviting you to a scheduled Zoom@IU meeting.
Topic: Routing Working Group Meeting 3 - Tool Talk - NetSage
Time: Jul 20, 2021 06:00 AM Indiana (East)
Join from computer or mobile:
https://iu.zoom.us/j/87951478476
Meeting ID: 879 5147 8476
One tap mobile
+13017158592,,87951478476# US (Washington DC)
+13126266799,,87951478476# US (Chicago)
Dial by your location
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
+1 669 900 6833 US (San Jose)
Meeting ID: 879 5147 8476
IU videoconferencing equipment: 26 879 5147 8476
Videoconferencing equipment outside of IU:
SIP: 87951478476(a)zoomcrc.com
H.323:
162.255.37.11 (US West)
162.255.36.11 (US East)
221.122.88.195 (China)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
209.9.211.110 (Hong Kong SAR)
64.211.144.160 (Brazil)
149.137.68.253 (Mexico)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 879 5147 8476
Zoom@IU Team | cthelp(a)iu.edu | https://kb.iu.edu/d/bfqu
Greetings Routing Working Group,
Reminder that we will have our next meeting tomorrow July 20 at 6:00am US Eastern time. This will be the next in our series of Tool Talks. Below please find the abstract for the talk on NetSage. You should have a calendar invite, however, I have attached an invite to this mail as well just in case.
Title- NetSage as a tool to find routing anomalies
Speaker – Jennifer Schopf
Abstract
The NetSage measurement and monitoring system (http://portal.netsage.global) is most often used by network owners to understand what data is using the circuits, what performance data transfers are experiencing, and for identifying changes of behavior over a set of links. However, it can also be used to identify end-to-end transfers that may not be taking the shortest path, especially in international settings.
This talk will give a brief overview of NetSage and its current deployments, then will show some real life examples of unusual routes – which may or may not be the preferred paths for data.
-----------------------------------------------
Hans Addleman is inviting you to a scheduled Zoom@IU meeting.
Topic: Routing Working Group Meeting 3 - Tool Talk - NetSage
Time: Jul 20, 2021 06:00 AM Indiana (East)
Join from computer or mobile:
https://iu.zoom.us/j/87951478476
Meeting ID: 879 5147 8476
One tap mobile
+13017158592,,87951478476# US (Washington DC)
+13126266799,,87951478476# US (Chicago)
Dial by your location
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
+1 669 900 6833 US (San Jose)
Meeting ID: 879 5147 8476
IU videoconferencing equipment: 26 879 5147 8476
Videoconferencing equipment outside of IU:
SIP: 87951478476(a)zoomcrc.com
H.323:
162.255.37.11 (US West)
162.255.36.11 (US East)
221.122.88.195 (China)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
209.9.211.110 (Hong Kong SAR)
64.211.144.160 (Brazil)
149.137.68.253 (Mexico)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 879 5147 8476
Zoom@IU Team | cthelp(a)iu.edu | https://kb.iu.edu/d/bfqu
Greetings Routing Working Group,
We will have our next meeting on July 20th at 6:00am US Eastern time. This will be the next in our series of Tool Talks. Below please find the abstract for the talk on NetSage. You should have a calendar invite, however, I have attached a calendar invite to this mail as well just in case.
Title- NetSage as a tool to find routing anomalies
Speaker – Jennifer Schopf
Abstract
The NetSage measurement and monitoring system (http://portal.netsage.global) is most often used by network owners to understand what data is using the circuits, what performance data transfers are experiencing, and for identifying changes of behavior over a set of links. However, it can also be used to identify end-to-end transfers that may not be taking the shortest path, especially in international settings.
This talk will give a brief overview of NetSage and its current deployments, then will show some real life examples of unusual routes – which may or may not be the preferred paths for data.
-----------------------------------------------
Hans Addleman is inviting you to a scheduled Zoom@IU meeting.
Topic: Routing Working Group Meeting 3 - Tool Talk - NetSage
Time: Jul 20, 2021 06:00 AM Indiana (East)
Join from computer or mobile:
https://iu.zoom.us/j/87951478476
Meeting ID: 879 5147 8476
One tap mobile
+13017158592,,87951478476# US (Washington DC)
+13126266799,,87951478476# US (Chicago)
Dial by your location
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
+1 669 900 6833 US (San Jose)
Meeting ID: 879 5147 8476
IU videoconferencing equipment: 26 879 5147 8476
Videoconferencing equipment outside of IU:
SIP: 87951478476(a)zoomcrc.com
H.323:
162.255.37.11 (US West)
162.255.36.11 (US East)
221.122.88.195 (China)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
209.9.211.110 (Hong Kong SAR)
64.211.144.160 (Brazil)
149.137.68.253 (Mexico)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 879 5147 8476
Zoom@IU Team | cthelp(a)iu.edu | https://kb.iu.edu/d/bfqu
