3864
Comments (240)
sorted by:
You're viewing a single comment thread. View all comments, or full comment thread.
24
mintscape 24 points ago +24 / -0

Anycast is not at all tricky, it can just get expensive as you really want your own AS number and would need your own address block. Used to be that anything less than a /19 was rejected by the backbone carriers and I highly doubt /19s are easy to get these days, especially for non-service providers.

I highly doubt T_D is taking any real advantage of anycast, scripts and other static things like images on their own CND URLS sure, but the main site is server created.

DDOS protection doesn't use Anycast anyway, it's a TCP reverse proxy, Cloudflare's meaty servers on huge pipes validate incoming connection attempts, they throw away the invalid sessions and pass the valid ones on to T_D servers.

14
AussieTrumpSupporter [S] 14 points ago +14 / -0

For the larger DDoS attacks you'd struggle without Anycast as your ISP links could flood if you can't segregate the traffic out. For basic ones it's pretty easy dependent on the level.

Expensive and tricky is much the same unless the donations flood in. Granted it's much easier to work with a site like TD than Gab given how much dynamic content is on Gab.

10
mintscape 10 points ago +10 / -0

Anycast is just advertising the same route out of multiple locations, it's an old trick for faster DNS response and for CDNs with static content and was in use long before it got the name Anycast.

For a dynamic service like T_D that operates on a single database, anycast can just not work. You would need to lock and synchronize the databases for every write from every location. It's impossible to maintain state without a huge impractical overhead that would make things unusable.

This why even places like Reddit handle their platform with load balancers on the front end and a huge database cluster.

I used to work for Cisco, I have done a huge amount of networking and worked on ios for them, I know what I am talking about my pede.

I get the feeling you actually don't know how Anycast works, it's just advertising the same route from different locations, nothing more. I'll explain.

Say I have a site in San Fran and I have a server 8.8.8.8, I advertise to my BGP peers 8.8.8.0/24. So there exists a single route to that server in the global routing table. Now people in Australia are using my server but complain it is too slow, so I add a new server with the same content and same IP address in Australia and I also advertise 8.8.8.0/24 from my Australian site to my BGP peers there.

So what happens? The global routing table ends up with two destinations to 8.8.8.8, what's the next hop? The route with the lowest cost. So Routers closer to Australia will pick the path to the Australian 8.8.8.8 server and those closest to San Fran will pick the path to the San Fran server. If one of those sites has a power outage, then I also get fail over automatically to the next closest server. I can keep adding locations and 8.8.8.8 servers all over the world if I like.

That's it, that's Anycast. It should now be clear why that can only work with sites or data that is static or gets updated very infrequently. It should be obvious that something like a dynamically built forum with hundreds of writes per minutes isn't suitable for Anycast. It is very common for reverse proxies to be on anycast addresses, this is what has people confused, as a reverse proxy just sends traffic to the real server.

3
AussieTrumpSupporter [S] 3 points ago +5 / -2

For the database for sure. I'm talking static content though. A DDoS often it won't even get to the database layer, it will just be flooding SYN packets. My point was moreso the fact that it would be hard for TD to build their own Cloudflare especially when they already have all the other complexities they deal with.

5
deleted 5 points ago +5 / -0
2
twopoint71 2 points ago +2 / -0

Could always run multiple load balancers and use DNS load balancing which confounds DDoS attacks pretty well. Anycast can get weird as the global topology shifts around. e.g. a client in a threshold area may start a connection to one location and might have to finish in another. An edge case to be sure, but not impossible. Anycast would require genuine statelessness to work effectively which is doable with with some good planning around session AAA.

2
deleted 2 points ago +2 / -0
1
mintscape 1 point ago +2 / -1

That's a good idea, I can tell you one thing about T_D, our mods know what they are doing, they are excellent. This site is so well run, it makes me smile as so many corporations don't come close to running things as well as our mods.

I'm a developer, got burned out from tech industry years ago but I am using my skills to help us and to start fighting back, it's a lot of fun but I have to be so careful.

-2
randomusers239874 -2 points ago +1 / -3

I used to work for a pretty big social media company. They all use anycast, not reverse proxies, with a multi-region setups. You don't need to have strong consistency on the data side of things, so you're incorrect. The typical setup is a pop ring equipped with a common anycast IP, with private interconnects to other locations. Updates are made to a local (or preferred) colo, then synced to the others. Because people tend to not move around to much you can rely on location to choose a preferred colo, and because this is not a live application, you don't need strong consistency between the colos. Each user seeing a slightly different thing doesn't matter since content is intrinsically ordered.

3
47urOFH3d 3 points ago +3 / -0

Wasn't IPv6 going to make address blocks easily available? Has anything happened with that in the last 10 years? (In case you happen to know).

6
mintscape 6 points ago +6 / -0

You are right, IPv6 was meant to solve the address depletion issue but it is extremely hard to get people to move to it. IPv4 is just what everyone knows and understands so they are trying to cling to it as long as they can. The main reason is addressing.

An IPv4 address looks like this 192.168.45.10, I guess you know that. An IPv6 address looks like this: 2607:f0d0:1002:0051:0000:0000:0000:0004

I can be shorted so the zeros are removed but it still looks like this:

2607:f0d0:1002:51::4

Then there is the problem of interconnecting Ipv4 to IPv6 networks, IPv6 is not backward compatible with IPv4 so you need to translate between the two and run both stacks on every router.

So people have spent effort in to delaying it, for example many mobile carriers run NAT between the mobile network and the Internet. If they didn't we would have been out of IPv4 addresses years ago because of smart phones.

So has anything happened in the last 10 years? Kind of, service providers are almost all running IPv6 in their core networks and many will give out IPv6 addresses on request but you generally need to ask for it. In the corporate world, mostly the same as things were 10 years ago, it's seen as an unnecessary change and I doubt many will even bother.

3
AussieTrumpSupporter [S] 3 points ago +3 / -0

Yep, laziness + NAT did this one to death.

2
47urOFH3d 2 points ago +2 / -0

Very enlightening. Thanks.

2
twopoint71 2 points ago +2 / -0

Yeah, IPv6 is basically free. The trouble is adoption is still not that great. I think all major wireless carriers use it now for mobile devices, so that should be good enough for most of us.

1
AussieTrumpSupporter [S] 1 point ago +1 / -0

Outside of the US they don't tend to. Mostly IPv4 with NAT.

0
SirPokeSmottington 0 points ago +1 / -1

Used to be that anything less than a /19 was rejected by the backbone carriers

That's when we were running out of address space. I think they've eased up now.

2
mintscape 2 points ago +2 / -0

The problem was the size of the global routing table, it got to the stage where the mid tier carrier class routers were getting close to running out of memory even when upgraded to max memory. So carriers refused to accept anything that wasn't summarized to at least a /19.

Likely changed now, those routers should be long retired by now.