A new exploit on SaltStack was actively exploited, resulting in the installation of malware on some rapyuta.io devices that were online during the time of the attack. Fortunately, the malware was a more uncomplicated variety, and we were able to remove it successfully from all affected devices.
We take data security and privacy very seriously on the rapyuta.io, and we are re-evaluating all internal components for security.
Technical Details
On April 30, F-secure discovered a new vulnerability found in SaltStack(CVE-2020-11651), which has affected other major companies as well. SaltStack is a popular tool used by major organizations. rapyuta.io uses it for managing devices on the platform. The vulnerability on SaltStack allowed for remote code execution on online devices on the platform.
On May 3, the exploit was actively exploited by a malicious party on rapyuta.io as well as several thousand other servers resulting in the installation of malware on rapyuta.io devices. We noticed the presence of the malware internally on May 4. The malware turned out to be a crypto-miner that starts using up all the CPU cycles of the device. It also adds a crontab entry to have persistence. We removed the malware from all active devices and also released the script we used to remove it as well.
Unfortunately, the patch on 29 April was missed by us, resulting in the vulnerability being exploitable for four days. The long weekend with engineers being away contributed to the delay as well.
Timeline
- 29-04-2020 – Public patches for the attack are pushed into SaltStack upstream.
- 03-05-2020 06:45:00 UTC – rapyuta.io devices are hit with the malware.
- 04-05-2020 04:40:00 UTC – Confirmed report of malware on rapyuta.io devices.
- 04-05-2020 06:40:00 UTC – Device Management is shutdown on rapyuta.io to prevent any additional payloads from being executed.
- 04-05-2020 13:30:00 UTC – All devices that were affected are successfully cleaned.
- 04-05-2020 15:04:00 UTC – Patched version is deployed, and device management is back online.
How it affects you as the user of rapyuta.io
We have reached out to all the users who were directly affected by this malware. If you did not get an email, you were not part of the affected group.
As a security measure, we have refreshed security and authentication tokens for all devices on boarded by users on our platform. As a result, all your devices will have to be re-authenticated with the rapyuta.io platform before you can use them again. We have preserved all the labels and metadata associated with a device.
To re-authenticate your devices, follow the steps below:
- Please select the device.
- Click on Token button.
- Copy the curl command
- Execute it on the physical device.
This only applies for older users. New users can follow the onboarding tutorial.
Measures taken by rapyuta.io for future security
We are making sure that all our internet-facing components are secured against generic vulnerabilities in the future. To prevent us from missing essential patches, we are making sure that we read and acknowledge patch notes/emails for all vital services used by rapyuta.io.
Security has always been an important priority at rapyuta.io. We have timely external third-party security testing of our platform done and also continue to work on improving the general security of the components.
Feel free to reach out to us at support@rapyuta.io for any questions or clarifications.
