Quickly Delete a Large Cloud Files Container

As you probably know, one cannot delete a Cloud Files container without deleting all the objects inside it first. This can take quite a bit of time if you have many objects in your container.

I had to delete about 30GB of data in a container and it was taking forever. I discovered a script by Fernando Florez which utilizes the Python Gevent library and Pyrax SDK to do a speedy container deletion. The Gevent library contains Greenlet which does micro-threading. Executing this script sends deletion requests to Cloud Files concurrently, drastically reducing total deletion time.

This method assumes you are utilizing a Python virtual environment and that you have activated it.

Install Pyrax

pip install pyrax

Install Libevent

curl -L -O https://github.com/downloads/libevent/libevent/libevent-2.0.21-stable.tar.gz
tar xzf libevent-2.0.21-stable.tar.gz
cd libevent-2.0.21-stable
./configure --prefix="$VIRTUAL_ENV"
make && make install
cd $VIRTUAL_ENV/..

Install Greenlet and Gevent

pip install greenlet
pip install gevent --global-option "-L$VIRTUAL_ENV/lib"  --global-option "-I$VIRTUAL_ENV/include"

Test Gevent

Enter a Python interactive shell and try to import Gevent. If you do not get any errors you should be good.

python
import gevent

Delete All Objects in Container

Be sure to specify your region, i.e. DFW, ORD, etc.

from gevent import monkey
from gevent.pool import Pool
from gevent import Timeout
monkey.patch_all()
import pyrax

if __name__ == '__main__':
pool = Pool(100)
pyrax.set_setting('identity_type', 'rackspace')
pyrax.set_setting('verify_ssl', False)
pyrax.set_setting("region", 'region')
pyrax.set_credentials('username', 'api_key')

cf = pyrax.cloudfiles
container = cf.get_container('container_name')
objects = container.get_objects(full_listing=True)
def delete_object(obj):
# added timeout of 5 seconds just in case
with Timeout(5, False):
          try:
              obj.delete()
          except:
              pass
  for obj in objects:
      pool.spawn(delete_object, obj)
  pool.join()

Install Pyrax

Install Libevent

Install Greenlet and Gevent

Test Gevent

Delete All Objects in Container

Comments