Service accounts can pose a security risk for your Google Cloud project if not managed properly. Because they are often highly privileged, anyone who is able to authenticate as a service account can likely take sensitive actions in your environment. It’s important to control access and have visibility into which identities — whether human or machine — have excessive and possibly dangerous permissions.
In this blog post, we’ll cover analyzing service account authentication patterns. You’ll be able to discover unused service accounts and keys and understand what API actions your service accounts are being used to take. If data access audit logs are enabled on your project, you’ll also be able to understand which users are authenticating via impersonation, the IPs that service account keys are being used to authenticate from, and determine which users originally created service account keys.
You’ll need the following IAM predefined roles to follow the steps discussed in this blog post:
Google’s Cloud Monitoring API allows you to view and query service account and service account key authentication events that occurred within the past six weeks. An “authentication event” is defined as any time a service account or service account key was used to call any Google API.
If you know which service account you are interested in, you can navigate to the “service accounts” page for your desired project here. You can then click on a specific service account and view its metrics.
You’ll be shown authentication traffic for the account, separated by impersonation events and key authentication events, as well as data on the API calls made from the service account.
You can also view aggregated metrics for all service accounts and keys via the Cloud Monitoring Metrics explorer, located here.
Use the drop-down and search to select either the “Service account authentication events” metric or the “Service account key authentication” metric.
“Service account authentication events” will include requests directly from an attached service account or requests made via service account impersonation, while “Service account key authentication events” will include requests authenticated with a specific key.
Google also provides various options for filtering, grouping, and aggregating the data.
You can group by
unique_id to view the data grouped by Service Account ID, or group by
key_id to view the data grouped by Service Account Key ID.
To query service account authentication events directly from the Cloud Monitoring API you can use the
projects.timeSeries.list API route (docs).
You’ll need to specify
metric.type = "
metric.type = "
" depending on whether you are looking for key events or direct authentication events. By default, this call will return raw time series data grouped by service account ID and service account key ID, but you can also request aggregated data via the
The Cloud Monitoring data provides us with a lot of insights into service account usage, but some details are missing. For example, who accessed these service accounts? Which of these accesses are human users and which are machines? To answer these questions, we can look to Google’s Data Access audit logs.
To view the data in this section, you’ll need to have had data access audit logs enabled in your project for the time period you wish to analyze. These logs are not enabled by default, but can be very helpful for analyzing access patterns in your cloud environment. See Google’s documentation for steps on how to enable data access logs.
One difficulty of understanding service account key usage is that there’s no way to know who has access to a key. Users may have downloaded it to their computer and shared it with others, which is something that you cannot track. However, you can make progress towards understanding which users have access to a key by determining who created the key, since Google only lets you download the key at creation time.
To do this, we can query the Logs Explorer to search for the
CreateServiceAccountKey event with the desired key ID.
You can narrow your search to the day the service account key was created by using the “Creation Time” field in the Cloud Console or returned by the IAM Service Account Keys API.
The resulting log will have an
authenticationInfo field that will tell you who created the key.
In the previous section, we covered obtaining timestamps for service account impersonation events. With logs, you can get more details on each event, including the principal that performed the impersonation and the specific request that was made.
Each audit log contains an
authenticationInfo field with information on how the request was authenticated. If the method was service account impersonation, there will be a
serviceAccountDelegationInfo field inside
authenticationInfo, which contains an array of all the principals in the impersonation chain. You can use this to query for all impersonation events.
If you want to query for all impersonation events involving human users, you can use the following:
protoPayload.authenticationInfo.serviceAccountDelegationInfo.firstPartyPrincipal.principalEmail =~ "@[YOUR DOMAIN]"
You can also query for all impersonation events involving service accounts via:
protoPayload.authenticationInfo.serviceAccountDelegationInfo.firstPartyPrincipal.principalEmail =~ "iam.gserviceaccount.com"
Filter for a specific service account by adding the following to your query:
Similarly, you can use the audit logs to get a more detailed log of service account key usage. Because there is no identity tied to the key usage, this unfortunately will not provide information on who used the key.
However, the audit log includes the IP address from which the request was made, which can allow you to further investigate the source of the key usage.
To search for all events on a specific key, you can use:
To search for events on any key, you can use the following:
protoPayload.authenticationInfo.serviceAccountKeyName =~ "//iam.googleapis.com/"
The resulting log will have a
requestMetadata field that contains a
callerIp field and a
callerSuppliedUserAgent field. You can use these to try to determine whether the key is being accessed programmatically or by a human user: for example, by cross-referencing the caller IP with your workload IP addresses.
It’s important to secure your service account permissions, because they may be allowing your users to have unintended access in your system and a common privilege escalation attack vector in Google Cloud is managing to authenticate as a privileged service account. Having excessively privileged service accounts in your infrastructure can leave you more vulnerable to both internal and external threats.
Here are some best practices to manage privileged access:
- Delete unused service accounts.
- Right-size service account roles. You should not have service accounts with “Editor” on your project. Instead, give them access to only the APIs that they’re being used to call. If you are using the same service account for a large range of functionality, instead consider replacing it with several different service accounts that each have more limited access.
- Delete unused service account keys.
- Avoid using service account keys unless absolutely necessary. Oftentimes, you can instead use service account impersonation or direct user credentials, both of which are more secure. See this flowchart from Google to understand alternative authentication methods you may be able to use.
Tired of having to check multiple Google APIs and querying logs in search for the information you need in order to secure your cloud configuration? Sign up for P0 for free and conduct your first IAM assessment in fifteen minutes or less. P0 will show you all your cloud principals (human and service accounts) along with all resources they have access to and information about their authentication history. P0 allows you to view service accounts and service account keys by usage easily, and will even alert you to changes in your security posture, such as new unused service account keys.