Orbit

Alo Anesi wrote:

All-

So from everything we discusses at the previous PACNOGs, it seems splitting up prefix advertisements is the best (only?) way to split inbound traffic when dealing with two different upstreams.

My question is, can I go as far as splitting at the /22 boundary for my /20? I could split at the /21 boundary, but with the way we've been allocating customer addresses 90% of my usage occurs in the first half of my /20.

I realize the CIDR folks may frown on de-aggregation, but I need some way to load balance two links of different sizes from 2 different upstreams.

I'm sure others will chime in here, but I think generally de-aggregation is frowned upon, but it sometimes can be the only way.

I've had reasonable success in the past by using a combination of heavy route prepending, and taking advantage of upstream providers communities and other TE knobs.

You may be able to do some load balancing by seeing if your upstreams support communities, and what communities they support. e.g. selective prepends to some of their peers (or all peers). You might also be able to deaggregate towards an upstream but with a no-export-to-peers community, meaning only their customers will follow the more specifics, and similar such tweaks.

aj.

Hi Alo,

Alo Anesi said the following on 14/7/06 20:00:

So from everything we discusses at the previous PACNOGs, it seems splitting up prefix advertisements is the best (only?) way to split inbound traffic when dealing with two different upstreams.

It's pretty much the only way unfortunately. No BGP implementer has support for the no-peer community (RFC3765); that would have helped with hiding local traffic engineering requirements of providers.

My question is, can I go as far as splitting at the /22 boundary for my /20? I could split at the /21 boundary, but with the way we've been allocating customer addresses 90% of my usage occurs in the first half of my /20.

Well, you could subdivide all the way down to a /32 if you really wanted to, but I really wouldn't bet anything on reachability of those specifics if you do that. It is really important to always announce your aggregate, even if you subdivide the block for traffic engineering purposes.

The other thing that very few people think about is how they do the address assignments within their own backbone. If 90% of the traffic comes into 5% of the address space, I'd encourage people to consider renumbering to make the load balancing easier. Dynamic assignment pools are easy to renumber, so if you need to move internal address space usage around to help with load balancing, that's one possible target (I did that years ago in my operational days).

I realize the CIDR folks may frown on de-aggregation, but I need some way to load balance two links of different sizes from 2 different upstreams.

As one of the "CIDR folks", I frown on unnecessary deaggregation. I reckon that 35% of the address space announced is simply people announcing their address blocks split unnecessarily into /24s - basically what I call leaking their internal BGP prefixes to the public Internet. Internal BGP is only meant to be visible inside an Autonomous System - external BGP has a different purpose, and that's where all SPs need to be prudent. So careful deaggregation to assist with traffic engineering is just one of those facts of life - but Alo, please don't take your /20 and spray it out to the world as 16 /24s with random AS-PATH prepends to do your load balancing. There are many better ways, e.g. as per the PacNOG workshop materials from the last meeting.

philip --

On 14-Jul-2006, at 06:00, Alo Anesi wrote:

So from everything we discusses at the previous PACNOGs, it seems splitting up prefix advertisements is the best (only?) way to split inbound traffic when dealing with two different upstreams.

Many people do this. Often it is the only mechanism available to do the kind of thing you're trying to do.

The downside, as Woody, Philip, Gaurab and Alastair have pointed out, is that you are contributing to the amount of information in the global routing table that everybody else has to pay for (e.g. in CPU, convergence time, etc).

As Gaurab pointed out, you might try adjusting the as-path prepending you're doing on your /20 first to see whether you can get a better balance of traffic without deaggregating. If that doesn't work (you may already have tried it), then some amount of deaggregation is going to be needed in order to attract traffic to the empty link.

In an ideal world, nobody would do it. In practice, however, I would say go for it; people who are upset by the extra routes can always filter.

My question is, can I go as far as splitting at the /22 boundary for my /20? I could split at the /21 boundary, but with the way we've been allocating customer addresses 90% of my usage occurs in the first half of my /20.

The only really widespread practice in filtering BGP advertisements from peers is to refuse prefixes which have a length greater than 24 (i.e. a /24 is the longest prefix you should expect people to accept).

Take your /20 and identify traffic sinks within it that fit into one or more covered prefixes (/21 to /24). We'll call these your traffic- engineering (TE) routes.

Advertise one of the TE routes to the ISP that you want to draw more traffic through, whilst still sending the /20 to both transit providers. Keeping the /20 route exactly as it is right now is important: it gives you the ability to experiment with different TE routes without the risk that someone will filter them and leave chunks of your customers without connectivity.

Wait a day, and see what that does to your traffic profile. If you still have room to fill on the backup transit provider link, try changing the set of TE routes you're sending them (e.g. send a second one as well, or send a /23 instead of /24). If the backup transit provider link is too full, withdraw the TE route you sent that way and try again with a different one.

This is a hit-and-miss, trial-and-error process, and if you want to keep your traffic as evenly-distributed as possible you'll need to repeat the exercise every so often.

Joe