> NervesCloud received 3389 simultaneous connected devices before the server hit me with the OOM killer. It was probably running a few more but around there. So each VM is: [...]
Is running a virtual machine per each connected device really necessary? Why do you do it?
I mean, I'm working at a IoT cloud too, but about 7 years ago we managed to connect 250.000 IoT devices to one erlang instance running on 8 core machine with 24gb ram as a stress test with simulated devices (rest of system was rabbitmq to share measurements from devices and riak for storing/retrieving them, plus websocket api endpoint). Those were gps trackers and some hubs. Surely you don't need 192 core server to connect 5000 devices?
lawik•6mo ago
No. It is not necessary. I don't think I said it was.
You can absolutely simulate more IoT-devices. But in this case I don't have to simulate receiving OTA updates, rebooting, keeping state. It has some upside but it isn't the reason I've done it.
I think I mentioned that in both this and the previous piece that I started out doing the VMs for fun and that's still kind of the central point. "What can I do with 192 cores?" We'll have a good use of the artifacts of this work for testing the framework itself in ways where devices are useful.
As you say, if I wanted to stress-test the OTA service for real I could definitely just set up some libraries to make multiple connections instead of starting the whole machinery. You can make a lot more connections, throw a lot more work at the system. And depending on how fancy you want to get you can simulate a lot of device behavior.
Given that I happened to have a couple of thousand full-fidelity devices in my pocket we got some incidental testing of the service.
yetihehe•6mo ago
Ohh, so you are testing end hubs as vm's on that server? Each instance is like another end device? That was hard to say from linked article and 5000 connections suggested that you are using vm for each connection on your iot server.
yetihehe•6mo ago
Is running a virtual machine per each connected device really necessary? Why do you do it?
I mean, I'm working at a IoT cloud too, but about 7 years ago we managed to connect 250.000 IoT devices to one erlang instance running on 8 core machine with 24gb ram as a stress test with simulated devices (rest of system was rabbitmq to share measurements from devices and riak for storing/retrieving them, plus websocket api endpoint). Those were gps trackers and some hubs. Surely you don't need 192 core server to connect 5000 devices?
lawik•6mo ago
You can absolutely simulate more IoT-devices. But in this case I don't have to simulate receiving OTA updates, rebooting, keeping state. It has some upside but it isn't the reason I've done it.
I think I mentioned that in both this and the previous piece that I started out doing the VMs for fun and that's still kind of the central point. "What can I do with 192 cores?" We'll have a good use of the artifacts of this work for testing the framework itself in ways where devices are useful.
As you say, if I wanted to stress-test the OTA service for real I could definitely just set up some libraries to make multiple connections instead of starting the whole machinery. You can make a lot more connections, throw a lot more work at the system. And depending on how fancy you want to get you can simulate a lot of device behavior.
Given that I happened to have a couple of thousand full-fidelity devices in my pocket we got some incidental testing of the service.
yetihehe•6mo ago