Cloud logging services provide comprehensive visibility into actions performed within cloud resources, making them essential for security monitoring. However, this reliance also makes logging services a high-value target for attackers. An attacker who exploits these services could create weak spots, evade detection, and in certain scenarios, establish continuous visibility within a target’s environment. Services such as Amazon Web Services (AWS) CloudTrail and Google Cloud are powerful for defenders, and prime targets for attackers seeking to remain undetected by disrupting the flow of logs. Attack techniques against cloud logging services primarily fall into two categories: Defense Evasion: Attackers aim to bypass detection systems, to execute attacks unnoticed. This may involve modifying resources within the cloud logging service. Continuous Visibility: Attackers attempt to transfer logs to their own accounts, establishing continuous visibility over the victim's environment. Understanding these attack scenarios enables organizations to implement the appropriate configurations and detect service misuse. Palo Alto Networks customers are better protected from the threats discussed above through the following products and services: Cortex Cloud The Unit 42 Cloud Security Assessment is an evaluation service that reviews cloud infrastructure to identify misconfigurations and security gaps. If you think you might have been compromised or have an urgent matter, contact the Unit 42 Incident Response team. Cloud Logging Services Serving as the authoritative system of record for every event, cloud logging services provide complete visibility into all actions within cloud environments. This comprehensive data enables analysis of past behaviors for both operational debugging and security investigations. Each cloud provider implements logging services in a unique way. Our recent Cloud Logging for Security article provides an overview of these various services across different cloud providers. In this article, we analyze and demonstrate attack techniques that target the primary logging services within each major cloud provider. Before examining the logging capabilities offered by major cloud providers, we outline the fundamental components and mechanisms for log delivery. Our analysis focuses on AWS CloudTrail and Google Cloud Logging. Both of these widely used services are designed to provide comprehensive audit trails and operational insights. While this article focuses on specific services, the attack techniques presented may also apply to other cloud logging services. Background: How Cloud Service Providers Handle Logs AWS CloudTrail AWS CloudTrail's primary resource for configurable log collection and delivery is known as a trail. A trail acts as a configuration that specifies how CloudTrail records AWS application programming interface (API) calls and related events in an AWS account. These events include actions taken by users, roles or AWS services. The main function of a trail is to deliver these captured logs to an Amazon S3 bucket. S3 is a highly scalable, durable, secure object storage service. When a trail is configured, it continuously writes log files containing event records to the designated S3 bucket. The S3 bucket serves as a centralized, long-term repository for CloudTrail logs. This enables auditing, security analysis and compliance efforts. CloudTrail supports sending native event trails to either the CloudTrail Lake feature, EventBridge or CloudWatch Logs. However, for enterprises with integrations to third-party products, these features might not be relevant or usable. Google Cloud Logging Cloud Logging is a fully managed service that collects logs from all of an organization’s Google Cloud resources. Cloud Logging leverages a resource called a sink as the primary mechanism for log delivery. A sink functions as a router that sends log entries to a specific destination. While a sink primarily routes and stores logs in centralized log buckets for analysis and retention, it also offers the flexibility to export logs to other powerful Google Cloud services. For example, logs can be routed to a cloud storage bucket for cost-effective long-term archival. When a sink is configured, it exports log entries that match specific criteria defined by a filter to a designated log bucket. Defense Evasion Sophisticated techniques enable attackers to remain undetected within a compromised cloud environment. These methods could involve manipulating or evading logging mechanisms, which are crucial for security monitoring and incident response. Obscured activities can extend an attacker's presence, facilitate data exfiltration or inflict more harm before being discovered. A wide array of security products including the following are fundamentally dependent on this log data to function: Security information and event management (SIEM) Security orchestration, automation and response (SOAR) platforms Specialized cloud security posture management (CSPM) tools By disabling, altering or deleting these logs, an attacker can effectively hide themselves from these defensive systems, compromising the security integrity of the entire cloud environment. Attackers often use the following five techniques to accomplish this: Stop logging Delete the log storage destination Delete the log router Impair logging via attacker-controlled encryption key Log poisoning Technique 1: Stop Logging The most direct method to suspend log flows is to disable the logging mechanism itself. A wide array of security products is fundamentally dependent on this log data to function. By disabling these logs, an attacker can effectively blind defensive systems, compromising the security integrity of the entire cloud environment. In AWS, an attacker with cloudtrail:StopLogging permissions can invoke the stop-logging API call for a specific trail. Once this API is executed, no further logs for that trail are written to the S3 bucket, creating an immediate visibility gap for organizations that created their own trails. In Google Cloud, the equivalent action is disabling the sink. An attacker with logging.sinks.update permissions can set the sink’s disabled field to true to stop logs from being written. The Google Cloud console message shown in Figure 1 reflects this change. Figure 1. Message confirming suspension of logs. Technique 2: Delete Log Storage Destination Typically, logs are stored in a cloud storage resource. An attacker who obtains permission to this resource could delete the cloud storage, preventing logs from being written. The following scenarios demonstrate this ability. In AWS, an attacker with s3:DeleteBucket permissions can use the delete-bucket API to delete the S3 bucket; this action also requires the s3:DeleteObject permission to empty the bucket. A few minutes after the action, the associated CloudTrail trail's configuration will indicate this deletion, as the notification in Figure 2 shows. Figure 2. An indication for the bucket deletion in AWS CloudTrail. In Google Cloud, an attacker with logging.buckets.delete permissions can delete the log bucket. In the case of a log bucket, the deletion is not immediate. Once the delete command is issued, the log bucket enters a DELETE_REQUESTED state, remaining in that state for seven days. After this period, the bucket is deleted. Google Cloud offers a mechanism to protect against this deletion by providing the ability to lock the logging bucket. Once a bucket is locked, its retention policy becomes permanent and irreversible, meaning the bucket cannot be deleted by any user until every log entry within it has fulfilled the specified retention period. Technique 3: Delete Log Router Another defense evasion tactic involves deleting the log routing resource – for example, an AWS trail or a Google Cloud sink. Once deleted, new logs will cease to be written to the designated destination. An attacker can delete a log router by using the delete-trail AWS API or the google.logging.v2.ConfigServiceV2.DeleteSink Google Cloud method. Technique 4: Impair Logging via Attacker-Controlled Encryption Key ​​An attacker could potentially render cloud logs unreadable by modifying their encryption key. An attack flow using AWS could unfold as follows: When a trail is created in AWS, one of the configuration parameters is the key management service (KMS) key, which is used to encrypt the logs delivered by CloudTrail. An attacker could prevent legitimate access to this key by updating the trail to use an attacker-controlled KMS key and then removing access to the key. Consequently, logs will not be written to the bucket due to the inability to encrypt them with the unusable key. To perform this attack, an attacker first creates an external KMS key with the policy shown in Figure 3 to ensure that the key is accessible to CloudTrail. Figure 3. Policy to create a KMS key with external access. The attacker uses the update-trail API to modify the KMS key used to encrypt the logs in the S3 bucket. Next, the attacker removes CloudTrail's access to the key by either deleting the key or removing the Allow CloudTrail to access the key statement from the policy. Subsequently, CloudTrail will indicate that there is a configuration issue due to denied bucket access, even though the bucket is correctly configured. From this point on, logs will not be written to the bucket because the KMS key will be inaccessible. Figure 4 shows the message that is displayed after access is disabled. Figure 4. Disabling access to the KMS key results in a Bucket access denied error. Figure 5 shows the attack flow using AWS. Figure 5. Impair logging via attacker-controlled encryption key attack flow in AWS. An attack flow using this technique in Google Cloud is as follows: In Google Cloud, a similar attack scenario can be simulated where a log bucket is already pre-configured with customer-managed encryption keys (CMEK). An attacker can then exploit this existing configuration by modifying the CMEK to reference an external key using the following command (the external CMEK must have encrypt/decrypt access granted to the KMS service account): gcloud logging buckets update BUCKET_NAME --location=LOCATION --cmek-kms-key-name FULL_KMS_KEY_NAME Subsequently, the attacker can remove the permissions granted to the external key. At this point, the victim will be unable to read the logs, as the Google Cloud panel in Figure 6 shows. Figure 6. The outcome of an inaccessible encrypted key. Any attempts to revert the key will result in the error message “rekeying requires that the CMEK service account has decrypt access to the current CMEK key,” as Figure 7 shows. Figure 7. The outcome of an attempt to revert the key. Technique 5: Log Poisoning Another defense evasion technique is the direct modification of logs – known as log poisoning. This is an effective technique when logs are pre-configured to be written to a cloud storage resource. In this case, the logs are stored in JavaScript Object Notation (JSON) format and can be modified by an attacker. If stored logs are deleted, added or modified, there is a high likelihood that Security Operations Center (SOC) personnel or analysts would inadvertently use these poisoned logs to conduct log analysis. In AWS, CloudTrail logs are delivered as objects to an S3 bucket. An attacker with s3:GetObject and s3:PutObject permissions on the bucket could download a log file, remove or alter specific events and then re-upload it, overwriting the original file. This breaks the chain of custody and invalidates the audit trail. Figure 8 shows the response to a query from a scenario where an attacker alters a log to avoid detection, and then the victim inspects the log using Amazon Athena. Figure 8. Athena query analysis shows the modified log. In Google Cloud, sinks route the logs to cloud storage. An attacker with storage.objects.get and storage.objects.create permissions can perform the same download and overwrite technique. To mitigate the risk of log poisoning, AWS provides CloudTrail log file integrity validation. This feature provides the ability to cryptographically verify whether log files were modified after they were delivered by CloudTrail. This ability is enabled by default when using the AWS Console to create Trails, but not when using the API or command line interface (CLI). Continuous Visibility Upon gaining an initial foothold in a victim environment, an attacker with advanced capabilities would aim to establish long-term, passive visibility into the victim's cloud infrastructure. Instead of running noisy discovery commands that might trigger alerts – or if they lack proper permissions – an attacker can target the log routing mechanism to route logs to their own environment, resulting in real-time visibility. This enables attackers to perform continuous discovery and passively monitor all activity, from new VM deployments and IAM policy changes to sensitive data access. In this way, attackers can map the environment, identify high-value targets, and escalate privileges while potentially remaining invisible to the victim's security monitoring. The following techniques achieve continuous visibility: Configure new log routing resource Log redirection Technique 1: Configure New Log Routing Resource A direct method for achieving continuous visibility involves creating a new log routing resource – for example, an AWS trail or a Google Cloud sink. The attacker configures the newly created resource to direct logs to an external, attacker-controlled destination. In AWS, the attacker configures CloudTrail logging to their own S3 bucket by using the create-trail API and specifying their bucket in the --s3-bucket-name parameter. In Google Cloud, the attacker utilizes the logging.sinks.create API to set the DESTINATION parameter to the attacker’s intended resource. Both of the above steps result in all logs being directed to the attacker's chosen destination. For certain AWS accounts, the CreateTrail operation shows up in CloudTrail. If EventBridge is configured upon AWS account set-up, defenders can use EventBridge to alert on creation events. In this setup, subsequent describe calls of CloudTrail configuration will show the attacker's destination bucket. However, for organizations that use third parties or have not applied these configurations, attackers can carry out adversarial activities without being detected. Technique 2: Log Redirection Using this technique, the attacker alters the log routing destination to one within their own environment. This redirects logs to an attacker-controlled resource, enabling the attacker to obtain continuous discovery. In AWS, the attacker updates the --s3-bucket-name parameter when invoking the update-trail API. After modifying the destination bucket, all logs are directed to the attacker’s bucket, providing continuous discovery capabilities on the victim’s account.

Source: Unit 42

Read Original Source →