These attacks are mostly possible because of the complacency of operators at many sites and companies. This is not a new problem and many of RFC's talk about methods for preventing and mitigating them, but most people don't care and prefer to just outsource everything to a single provider, which becomes the weakest link.
The Internet wasn't envisioned with a single email provider, single DNS provider, single app container provider. (Ok, for most of these you have two, sometimes three choices, but still, that is too few). The centralization makes everything very vulnerable - imagine what would happen when Gmail is knocked out for a day.
Seriously? It's OK if only one site/company gets taken offline at a time?
There's no RFC that talks about methods for preventing or mitigating hundreds of thousands of machines all sending arbitrary traffic at you at the same time.
The only way to protect yourself from that sort of attack is to buy filtering from someone who has a bigger pipe than the largest DDoS available, and have them filter the packets so that you only get clean traffic. Unless you know of an alternative that nobody else has heard of yet.
So you wind up buying transit / scrubbing from one of a few big providers, because that's the only way to avoid being sniped by DDoSers.
> There's no RFC that talks about methods for preventing or mitigating hundreds of thousands of machines all sending arbitrary traffic at you at the same time.
The RFCs generally say that the problem is "you", i.e. the target. Of course those device makers could make their devices a little more secure, can't argue with that, it's another form of complacency. Still - the attackers are only able to do this because their targets are few.
If there were thousands of DNS providers such as Dyn each serving a small number of clients spread all over the world, it'd be impossible to attack them all.
To cause maximum damage you need to identify hosts that are common across many big companies. Someone did their homework and figured out that lots of companies are using Dyn for DNS, and for the East Coast of the US this is just a handful of servers. If the same DNS services were spread across 1000 servers, then the attackers would need proportionally more "power" to knock them out. DDos-ing 10 boxes is _so_ much easier than 1000 (approximately 100 times easier, to be precise).
The problem with these devices in particular is the weak point is the user. As is the case in most attacks.
Your average user says "Sure I can setup cameras" then sees "remote access" in the menu, sets it up, maybe it has some UPNP to the router and BOOM. Magic remote login without any type of mitigation.
Indeed. My mom got an internet connected "security camera" kit (for cheap from one of the big wholesalers, can't remember the manufacturer) and asked me to set it up.
The hardware was nice, cameras did a reliable 1080p full color, but the whole reason my mom wanted it was so she could check in while she and my dad were traveling (and also sneak a peek at her bird feeders while she was away; avid birder, that one).
So, I hooked that thing up to the network and did a port scan on it... First noticed - it's listening to port 22, auth is a googleable default password. It supports UPnP to punch a hole through the NAT and serve up video on another port. OS on the server box is some slightly customized version of linux with an _old_ kernel.
So I said, "Sure mom, I can set this up for you. We're going to need to get you a new firewall, it'll probably be easiest to put a *nix box in front of your wifi access point, then we can set up a tunnel between the isolated camera server and a locked down outside server that only you have access to so we can be sure that no one else is looking at those cameras. Should only take me a few hours, and we'll need to buy a box to run the firewall, and then a small monthly fee to keep the internet accessible server running"
Her response, "but it says on the box that it's easy to setup for outside access!". Mine: "It's easy to setup for everyone to access, much more involved if you want to make sure it's only you who has access".
Admittedly, it did have some authentication for accessing the video streams, but I didn't trust that thing as far as I could throw it; I'm glad she decided not to go through the trouble of getting it working (but mostly because I'm lazy and didn't want to have to setup and support that damn thing).
I can only imagine that the people who bought that device and didn't have a security paranoid person to help them set it up are all contributing to this most recent DDoS attack.
Her response, "but it says on the box that it's easy to
setup for outside access!". Mine: "It's easy to setup for
everyone to access, much more involved if you want to
make sure it's only you who has access"
well that was a pretty clever answer, I needed to laught about that :D
Basically the commercial was right :D "easy to setup for outside access" that didn' implied a single person ^^
The problem here is there's nearly zero incentive to do it right. I mean, ok, let's say the worst - somebody breaks in the box. For a regular person, worst thing somebody would get access to their DVR. As long as it keeps working as DVR, they couldn't care less. Yes, this DVR would also serve as botnet bot, but the owner doesn't care. It doesn't hurt them - except when Twitter goes down but they don't make the link between them not configuring the DVR properly and Twitter going down. Until we find a way to make the incentives work in right direction, nothing really would change...
It may not have been over HTTP, so possible to be sniffed. Or, even if it did have HTTPS, it might not generate keys in a secure way (or might use the same certificate as other devices). And you don't know if there are hidden backdoor accounts that might be found eventually...
So, yeah, it makes sense to block it - personally I block IOT devices from the Internet entirely (and don't let them initiate requests to my local network even) and use a VPN (IPSEC/IKEv2). That wouldn't work for devices that connect to cloud services, so I'd have to set up new firewall rules if I got one of them.
Late response, but yes - there was no https support whatsoever on this thing. Authentication was some custom shit and intended to be passed over the internet in clear text.
The real problem here, and this isn't going to be a popular position, is that you're relying on the internet for important things.
The original engineering and architecture of the the internet (and the web) was not intended to create something you put all your eggs in. It was for sharing information, not building your mission critical business operations on.
Right now, if you dumped your business into a cloud service you're mostly dead in the water. But those who have local infrastructure can keep working. As people have been noting here, centralization is bad.
Actually, the original engineering and architecture of the internet was intended to provide reliable command & control in the event of a nuclear war. A network of last resort. I can't think of anything more mission critical than that.
No, it wasn't. That's a myth, disturbed in many sources, including [1]. Also in [2]:
Many people have heard that the Internet began with some military computers in the Pentagon called Arpanet in 1969. The theory goes on to suggest that the network was designed to survive a nuclear attack. However, whichever definition of what the Internet is we use, neither the Pentagon nor 1969 hold up as the time and place the Internet was invented. A project which began in the Pentagon that year, called Arpanet, gave birth to the Internet protocols sometime later (during the 1970's), but 1969 was not the Internet's beginnings. Surviving a nuclear attack was not Arpanet's motivation, nor was building a global communications network.
Bob Taylor, the Pentagon official who was in charge of the Pentagon's Advanced Research Projects Agency Network (or Arpanet) program, insists that the purpose was not military, but scientific. The nuclear attack theory was never part of the design. Nor was an Internet in the sense we know it part of the Pentagon's 1969 thinking. Larry Roberts, who was employed by Bob Taylor to build the Arpanet network, states that Arpanet was never intended to link people or be a communications and information facility.
Where Wizards Stay Up Late is a fairly dry book, but it contains interesting kernels of information (like this). It's not a page turner, but it's worth a read if you are interested in things like, for example, the information in this comment's parent.
> Arpanet was about time-sharing. Time sharing tried to make it possible for research institutions to use the processing power of other institutions computers when they had large calculations to do that required more power, or when someone else's facility might do the job better.
Arpanet is distributed shared information for science. Nuclear technology is science. Surviving science is a war that requires nuclear insights. Therefore, the Arpanet was developed for surviving nuclear war.
Exactly, I have tons of IOT devices. I put them on a separate subnet that does not have a gateway to the internet then I VPN into that network to access them. Perhaps a product that makes that a simple process will solve the problem?
We partly do that at Wormhole. I say partly because you still have to be able to access one of our addresses. Port of last resort is 443/TCP, so it works on lots of tricky networks out there.
The idea is that all your IOT stuff establishes a connection to this server, creating an encrypted network between them. You then add your control servers to that network and job done.
You devices don't need any inbound access to talk to each other. All the connections are outbound, so no ports to open on your firewall and no risk.
You could do this by yourself, but we take that hassle out of your hands. Happy to help with custom deployments too outside our main service; it's a great way of learning our customers' needs.
It's hard though to have your exact setup as a service, it implies incoming VPN connections to the site where you deploy your IOT and a VPN server of sorts.
Our main focus was remote teams and devs having to use remote servers, however IOT might be a killer use here.
Interesting, I have a few thoughts. Perhaps you could sell a preconfigured pfsense box (or make a raspberry pi image to start with) that when plugged into the customers router creates a reverse tunnel via your service as well as a WiFi hotspot. Then offer the user a very simple firewall control panel and they can choose what devices to allow to the open internet and what to keep private and accessible via some sort of authenticated channel. Thus devices that contain sensitive data or require enhanced security (cameras, private network attached storage devices, home automation) and devices that require internet access (Amazon Echo) can both be served by the service.
Very nice service by the way. I have used ngrok in the past and found it invaluable for a few odd applications. I'll give it a try in future.
Thank you for the feedback and the suggestion. It is a good idea actually. I'm considering new features in the roadmap, because at the moment I don't even offer Internet access through my system, it's just a private LAN (I'm not competing with the myriad of privacy-minded browsing VPNs out there). Adding a manageable Internet Gateway could be a nice option.
Developing and deploying a software+hardware piece would be very interesting too, so there's no need to deploy agents on the remote servers or IOT devices (on most of them you probably can't) and I take the hassle out of my customer's hands to setup a e.g. Linux gateway to route traffic through the tunnel.
A flexible gateway would be a great add on, I also like a private DNS server while developing. If you offered a Postfix forwarder and static, clean IP addresses, you could attract home users who wish to host their own email but are behind dynamic residential connections (like me, I use a digital ocean droplet currently for that purpose).
FWIW, I would definitely be interested in paying for a service like this. I'm technical enough to care about this, but not technical enough to solve it myself. Similar to where I was before dropbox.
It could suit your needs or we can help with custom deployments. In any case I'd like to learn more about your needs and your expectations. Can I drop you an email?
I've been thinking about how you'd design a UI for that, that was easy to use. Maybe a separate wifi network that IOT devices go on to, and then a web app that knows devices with XYZ MAC are LIFX bulbs and shouldn't be able to talk to the smart TV, but that phones on the network should be able to jump the subnet and talk to the bulbs.
You can make it semi-automated in a way. I believe the first 6 characters of the mac address are the vendor id, I'd get the DHCP server to assign different vendors into different isolated vlans but with short leases at first and then allow you to merge them, assign permissions and move them around. Call it "learning mode". It won't be perfect but you can also augment it with human created presets.
The problem with any solution is getting it used widely enough to make a difference. We seem to have an unlimited predilection for making the same mistakes repeatedly, even though we could avoid them.
The Internet wasn't envisioned with a single email provider, single DNS provider, single app container provider. (Ok, for most of these you have two, sometimes three choices, but still, that is too few). The centralization makes everything very vulnerable - imagine what would happen when Gmail is knocked out for a day.