DATADOG got hacked?

4 points

10 years ago

We've faced a nightmare weekend and ran into a locked account at AWS. After more than two days they told us that our key at DATADOG got compromised?

Hello XXX,

Thank you for reaching out for your patience. I’m reaching out to you in behalf of my colleague David as he is currently off shift. Regarding this issue the EC2 team have new information, and they mentioned that spot instance requests are block due to a compromised key in your account. An email regarding this issue was sent on Tuesday, July 12, 2016 at 4:26 PM PDT from [email protected] to the email [email protected] whit the following Subject “Informational Message Regarding Security Incident at Third Party ‘Datadog’ [AWS Account: X]”

Here is the information regarding the compromised key in your account XXX: Type: Access Key Pair Credential: XXX IAM User: datadog

To rotate access keys, you should follow these steps: 1. Create a second access key in addition to the one in use. 2. Update all your applications to use the new access key and validate that the applications are working. 3. Change the state of the previous access key to inactive. 4. Validate that your applications are still working as expected. 5. Delete the inactive access key.

Here are some resources that you might find useful: [1] https://blogs.aws.amazon.com/security/post/Tx15CIT22V4J8RP/How-to-rotate-access-keys-for-IAM-users [2] http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#rotate-credentials

Once these actions are taken please update the case so we can reach out to the EC2 team and they can remove the blocking for spot EC2 instances. If you have any questions or concerns regarding this issue please let us know we will happy to further assist you. Thank you!

XXX Amazon Web Services

Are there anybody else out there with the same issue? Maybe the DataDog team could provide their perspective? We're now sitting on the additional AWS costs.

4 comments