Skip to main content

S3 - Using S3cmd

Our Object Storage service supports the Amazon S3 protocol, allowing you to use standard S3 tools and SDKs to manage your data. This guide explains how to use S3-compatible features, create credentials, and interact with your storage using the s3cmd client.

caution
info
  • EC2 Credentials: EC2 Credentials are associated always to a user, if an application needs to be using the credentials, ensure a svc-user own the ec2 created credentials. Only this user will be able to retrieve the credentials.
  • Region Setting: Some software requires a region. Use Valencia as the default. All data is stored in our data centers; this is only for compatibility.
  • Path Style: If using an S3 SDK, set forcePathStyle=true to avoid issues with bucket operations.

Create S3 Credentials​

Generate Openstack ec2 Credentials​

To use the S3 protocol, you need an access key and secret key. Generate them with:

openstack ec2 credentials create

Example output:

access a2368e891f24xxxxxxxxxxxxxxxxxxxx
secret 4329f35488954xxxxxxxxxxxxxxxxxxx
user_id 9b3c7f1a2d4e5f6081723a9b0c1d2e3f

Save your access and secret keys securely.

List Existing Credentials​

openstack ec2 credentials list

S3 Endpoint​

Use the public endpoint for S3 API calls:

https://s3.uniccloud.org/

An internal endpoint is also available:

https://radosgw.api.eu-vlc.uniccloud.org/
caution

if using the internal endpoint, you must use UNICC CA certificate installed otherwise use flag --no-check-certificate

Using the s3cmd Client​

Configure s3cmd​

Sample Configuration File Here’s a sample .s3cfg file:

access_key = 
secret_key = 
host_base = radosgw.api.eu-vlc.uniccloud.org:443
host_bucket = tf-test-container.radosgw.api.eu-vlc.uniccloud.org:443
use_https = True

Managing Object Storage with s3cmd​

Create a Bucket​

s3cmd -c .s3cfg mb s3://BUCKET

List Buckets​

s3cmd -c .s3cfg ls

Example output:

2018-11-19 13:02  s3://BUCKET

Upload Files​

s3cmd -c .s3cfg put ~/test.json s3://BUCKET

Example output:

upload: 'test.json' -> 's3://BUCKET/test.json'  [1 of 1]
 3511 of 3511   100% in    0s     7.68 kB/s  done

Set Object Properties​

s3cmd -c .s3cfg put --mime-type='application/json' --add-header='Cache-Control: max-age=3600' ~/test.json s3://BUCKET

List Files in a Bucket​

s3cmd -c .s3cfg ls s3://BUCKET

Example output:

2018-11-19 13:07      3511   s3://BUCKET/test.json

Delete Files​

s3cmd -c .s3cfg del s3://BUCKET/test.json

Example output:

delete: 's3://BUCKET/test.json'

Delete a Bucket​

Note: The bucket must be empty.

s3cmd -c .s3cfg rb s3://BUCKET

Example output:

Bucket 's3://BUCKET/' removed