Skip to main content

S3 - Lifecycle Policies

A lifecycle policy tells Object Storage to expire the objects of a bucket automatically after a given number of days. Retention is then enforced by the storage platform itself, instead of by a scheduled job you have to write, run, and monitor.

Lifecycle policies are part of the S3-compatible API, so they are managed with an S3 client such as s3cmd.

caution
  • Expiration is permanent. Expired objects are deleted, not moved to a recycle bin.
  • A rule applies to the objects already in the bucket, not only to the ones uploaded after it was set.
  • <Days> is counted from each object's last-modified time, so objects older than the retention period start expiring as soon as the rule is enabled.
  • On backup or archive buckets, confirm your retention requirement before enabling a rule.

Prerequisites​

  • An access key and a secret key, and a working s3cmd configuration file. See S3 - Using S3cmd for how to create the credentials and write the .s3cfg file.
  • The commands below use -c .s3cfg to point at that configuration file.
info

The configuration file holds your secret key. Keep it out of version control and out of shared directories.

Writing a Rule​

A policy is an XML document containing one or more rules. The rule below expires every object in the bucket 120 days after it was last modified:

<LifecycleConfiguration>
  <Rule>
    <ID>expire-objects-in-120-days</ID>
    <Prefix/>
    <Status>Enabled</Status>
    <Expiration>
      <Days>120</Days>
    </Expiration>
  </Rule>
</LifecycleConfiguration>

Save it as, for example, expire-objects-in-120-days.xml.

Rule Requirements​

  • Always set an explicit <ID>. If you omit it, the gateway generates a random 48-character identifier, which makes it impossible to confirm later that your policy is the one in place.
  • Use <Prefix/> rather than <Filter>. This is the form our gateway accepts. Leave it empty to cover the whole bucket, or scope the rule to a path with <Prefix>logs/</Prefix>.
  • Every rule needs at least one action element. A rule that carries only a <Status> is rejected.

Managing Lifecycle Policies with s3cmd​

Apply a Policy​

s3cmd -c .s3cfg setlifecycle expire-objects-in-120-days.xml s3://BUCKET

Example output:

s3://BUCKET/: Lifecycle Policy updated

Read the Current Policy​

s3cmd -c .s3cfg getlifecycle s3://BUCKET

Example output:

<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Rule>
    <ID>expire-objects-in-120-days</ID>
    <Prefix/>
    <Status>Enabled</Status>
    <Expiration>
      <Days>120</Days>
    </Expiration>
  </Rule>
</LifecycleConfiguration>

Verify Every Write​

Always run getlifecycle after setlifecycle, and check that the <ID> you sent is the one that comes back. If it is not, the policy you intended is not the policy the bucket is enforcing.

Stopping an Expiration Rule​

To stop objects from expiring, replace the policy with a disabled rule instead of removing it. A disabled rule still carries an <Expiration> element, because a rule without an action is rejected; the value is never applied while the rule is disabled.

Save the following as lifecycle-disabled.xml:

<LifecycleConfiguration>
  <Rule>
    <ID>disabled-no-expiration</ID>
    <Prefix/>
    <Status>Disabled</Status>
    <Expiration>
      <Days>22000</Days>
    </Expiration>
  </Rule>
</LifecycleConfiguration>

Apply it and confirm the change:

s3cmd -c .s3cfg setlifecycle lifecycle-disabled.xml s3://BUCKET
s3cmd -c .s3cfg getlifecycle s3://BUCKET

The returned policy must show <ID>disabled-no-expiration</ID>. If the identifier does not change, the bucket is still enforcing the previous rule — stop and contact the UNIQCloud team at [email protected] before relying on the bucket.