Earlier this week, Secure Ideas sent an initial notification regarding an incident targeting us that took place at a vendor. The initial notification email is available at: https://training.secureideas.com/newsletter/aom-incident-notification/).We promised at that time to release more details as soon as we collected them and better understood the situation. In this blog post, we share what we have learned about the mechanism of the privilege escalation, and how we have been working to resolve it.
We would also like to take a second and thank Academy of Mine for their awesome response when we contacted them. Academy of Mine is the vendor for our Learning Management System (LMS) at https://training.secureideas.com
On the morning of Friday, March 22, Secure Ideas discovered that two student users had gained administrator privileges within our training site, training.secureideas.com. This is a multi-tenant, third-party, hosted, and managed system. We say this not to deflect any of the concern or blame, but to explain how the system works. These two people had gained a level of access above what Secure Ideas has within the system. This prevented us from removing their access or the accounts. This is often the case in multi-tenant systems as the organization that runs the systems tends to have a level of accounts above their clients. In our case, we are considered Client Administrators and the attackers had elevated to Administrator.
Our investigation is still in the early stages, but it is clear that the attackers took advantage of a vulnerability in a WordPress plugin called Woocommerce. The vulnerability allowed the attackers to use a purchase of training course materials to elevate their user accounts to the admin role. We believe that this exploit is related to the one announced late last year. Our vendor uses an extremely customized version of WordPress and Woocommerce and thought they had fixed the issue previously.
Secure Ideas, along with the support of Academy of Mine (AoM), has taken immediate action to better protect our students. We have moved the Secure Ideas training site into seperate server and disabled web access to the Administrator role. We have also installed additional security features to safeguard user information and more effectively monitor usage. We are also working with AoM to add significant other controls that we won’t be discussing here.
The following is a timeline of the first hours of Secure Ideas’ investigation into how the breach occurred, as well as the communications with the AoM support team. We are publishing this information to provide additional technical details that was not covered in previous notifications and to show the transparency that is integral to our principles as an organization. Secure Ideas has long believed that full disclosure in the event of a security incident is foundational to our customers’ security state of mind. We hope that by providing the following sequence of events, we can effectively practice the same guidelines that we give our customers and inspire the confidence that we are doing our best to strengthen our services against future incidents.
[2019-03-22 08:45 EST] Kevin discovers the additional administrator accounts. At the time of discovery, there is no way to remove the role, or remove the users. Academy of Mine (AoM) is notified of the issue via a ticket on the helpdesk page. |
[2019-03-22 08:51 EST] Kevin calls a meeting and informs Secure Ideas staff of the incident.
[2019-03-22 09:03 EST]Kevin assigns members of the Secure Ideas team with reconnaissance and mapping of the AoM system. Kevin reminds these members that Secure Ideas doesn’t have permission to test AoM. They are advised not to do anything that could harm the AoM’s production environment.
[2019-03-22 09:05 EST]AoM Support contacts Secure Ideas via Helpdesk. AoM Support states that the users are genuine. AoM Support suspects that the 2 Factor authentication (2FA) plugin recently added has a bug and that is related to how the users got the admin role. AoM Support also states that the users have not done anything malicious so far. AoM Support has decided to disable the 2FA plugin until further analysis is done.
[2019-03-22 09:14 EST] Kevin is on a call with AoM support team. He requests that an audit plugin be installed so that Secure Ideas can evaluate their logs, he requests the time and date that the two users granted themselves admin rights. AoM Support suspects there was a bug in the 2FA plugin that they installed at Secure Ideas request the week previous.
[2019-03-22 09:39 EST] AoM Support installs the audit log plugin. They are also in the process of adding a piece of code in the backend which will not allow any user to get administrator rights without manually being assigned in the database.
[2019-03-22 10:24 EST] Kevin contacts AoM Support via the helpdesk. He is requesting logs that detail what the users did over the last 80 hours. He states that Secure Ideas is trying to determine if they accessed admin functions, users lists, or changed any content.
[2019-03-22 10:46 EST] Secure Ideas begins the process of documenting the incident and the communication with AoM Support. The breach notification email is being drafted.
[2019-03-22 11:49 EST] Kevin contacts AoM Support via Helpdesk to ask when access was granted by the system and to troubleshoot access problems with the audit plugin.
[2019-03-22 13:11 EST]AoM Support fixes the audit log plugin access. AoM Support has found that a purchase of a course through a Woocommerce plugin resulted in users being granted admin access. AoM Support provides specific order numbers. They have also done an analysis of the database backups from the last 60 days and believe that the attackers did not do anything after they got access. AoM Support announces that the Secure Ideas training site will be set up on a separate server and Secure Ideas will be granted a new level of access.
[2019-03-22 13:32 EST]After Kevin contacts AoM management on the phone, he informs the staff of the following: AoM will share system logs regarding the event, AOM has determined that Woommerce is the plugin that allowed the privilege escalation, and that according to AOM, the users escalated privileges and then never logged in again after that. Kevin also informs staff that AOM wants to move SecureIdeas to a Single Tenant system to allow SecureIdeas to install WordFence as well as other security features. AOM will also allow system level access complete with system logging information.
[2019-03-25 08:43 EST]AoM Support states that they would add backend code today to ensure no one can become administrator even if they try to use a hole in some plugin and the developers are testing that out right now.
[2019-03-25 11:34 EST]2FA plugin is reenabled.
[2019-03-25 14:17 EST]Kevin notifies the users of training.secureideas.com of the security incident via email.
[2019-03-25 14:26 EST]Kevin notifies the users of training.secureideas.com of the security incident with a message on the Professionally Evil Slack.
[2019-03-25 14:29 EST]Kevin notifies the users of training.secureideas.com of the security incident with a post on the company Facebook, Twitter and LinkedIn accounts.
Secure Ideas is continuing to investigate the details of the incident along with AoM. This post will be updated as more details become available.
Our goal in making this post is to demonstrate what a good incident notification should look like. While no one ever wants to be in this situation of having to write an incident report, we hope that our response can be used as a model for others in the future. We’re serious about our efforts towards industry training and hope that our failure can stand beside our successes to improve the community.
If you have questions about this incident or need more information, feel free to contact us at firstname.lastname@example.org.