Static is always a topic in Django projects that it requires careful settings before the projects can work smoothly.
In Django, static refers to a collections of static files or a static folder which stores the common settings of your web applications such as CSS, Javascripts. Usually we will have the
static
folder put right inside the project folder, at the same level of
manage.py
. By calling
collectstatic, Django gathers the static files from this folder before running the project such that it knows where to lookup when you use the tag
{% load static %}.
0.
[Advantages]
The advantages of using AWS S3 include but not limit to:
- Cloud storages which means you can save more spaces for your project
- Secure online storages with AWS IAM which we will talk about later
- Unlimited spaces for uploads
1.
[Register and IAM]
First of all, register an AWS account to use this amazing service. Note that this is not the same as your amazon account for shopping.
- Go to https://aws.amazon.com/free/ and create an account.
Once you are registered and logged into the console, open the Services tab on the left top corner, and find a service named IAM under 'Security, Identity & Compliance'. IAM is short for 'Identity and Access Management' where you can create groups or users to use your AWS services. Instead of using your AWS root account, the benefit of using IAM group+user is that you can control their permission to use your AWS services. Furthermore, it also helps to protect your files that every user needs credential key ID + secret key to access (i.e. programmatic access). Read their documents here: https://aws.amazon.com/documentation/iam/ for more detail.
- So in the IAM page, click 'Users' tab and 'Add User'. Give it a username such as 'django-user' or anything you prefer. Check the 'Programmatic Access' checkbox and proceed.
Now, we need to create a 'Group' for that 'django-user' in order to give it rights to access our coming AWS S3.
- Click 'Create Group' and assign a name e.g. 'django-group'. Search for the policy 'AmazonS3FullAccess' and check the box. Click 'Create Group' and 'Next: Review'.
In this page, you will found the
Access key ID and
Secret Access for this group of user(s). You may cap-screen for the detail but if you forgot to do so, no worries because you can find them again from the
Users tab.
2.
[Set up S3]
After creating
IAM, we are now going to register the de facto AWS S3.
Click 'Create Bucket', assign a name for your S3 bucket(the cloud storage), e.g. 'django-s3'. Choose the region you preferred then click 'Next' to finish.
3.
[Install packages]
There are 2 packages you will need for assigning the storages option.
4.
[Django Settings]
Here comes the main part. In this session, we need to let Django knows our 'static' location is now moved to AWS. To do so, we need to connect AWS via the 'group&user' created earlier.
- In
settings.py
, head to your 'static' settings and replace it into these:
(Following keys are not real, only resemble)
#AWS S3 connection
AWS_ACCESS_KEY_ID = 'HASBDJASBDBBNBDSLFHA'
AWS_SECRET_ACCESS_KEY = 'bhasd7ASVDLASJDB9ADNASDBLSDBJASDB+SDNads/ASDHJ'
AWS_STORAGE_BUCKET_NAME = 'django-s3'
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
Now this part tells how Django should connect to AWS S3. Note that those private data such as ID, keys and name should be enclosed in .env
file by using Python-Decouple as mentioned in previous posts.
- Next we have to tell it where and in what manner should it store the 'static', 'private media' and 'public media' files.
#AWS S3 Static
AWS_STATIC_LOCATION = 'static'
STATIC_URL = 'http://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_STATIC_LOCATION)
STATICFILES_DIR = [os.path.join(BASE_DIR, 'static')]
STATICFILES_STORAGE = '<YOUR_PROJECT_NAME>.storage_backends.StaticStorage'
#AWS S3 Private Media Upload
AWS_MEDIA_LOCATION = 'media'
PRIVATE_FILE_STORAGE = '<YOUR_PROJECT_NAME>.storage_backends.MediaStorage'
#AWS S3 Public Media Upload
AWS_PUBLIC_LOCATION = 'public'
DEFAULT_FILE_STORAGE = '<YOUR_PROJECT_NAME>.storage_backends.PublicStorage'
Here we tells django we are going to use 3 different folders inside the AWS S3 bucket, 'static', 'media' and 'public'. Such that static files such as CSS and JS will be kept in 'static', files uploaded by you/web admin will be stored in 'media', and visitor may upload files into 'public' folder.
- Then we have to create our python file
storage_backends.py
so that the snippet above can make use of.
create a new file named as storage_backends.py
and save it next to settings.py
. Make the content looks like this:
from django.conf import settings
from storages.backends.s3boto3 import S3Boto3Storage
class StaticStorage(S3Boto3Storage):
location = settings.AWS_STATIC_LOCATION
class MediaStorage(S3Boto3Storage):
location = settings.AWS_MEDIA_LOCATION
default_acl = 'private' #To turn access control list into private use only. Will use it in models.py
file_overwrite = False #Not to replace files even they have same name
custom_domain = False
class PublicStorage(S3Boto3Storage):
location = settings.AWS_PUBLIC_LOCATION
file_overwrite = False
5.
[FileField or ImageField in models]
By this far, the settings are almost ready. The things that you still have to amend is
models.py
where you need to tell which field you are going to make it 'private upload only'.
"""
import dj_database_url
DATABASES = {
'default': dj_database_url.config(
default=config('DATABASE_URL')
)
}
"""
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'database_name',
'USER': 'postgres',
'DBPASSWORD': config('DBPASSWORD'),
'HOST': '127.0.0.1',
'PORT': '5432'
}
}
Make sure local database is running, open terminal and collectstatic:
$pipenv run python manage.py collectstatic
This takes much longer than it usually does as it uploads to AWS. Once finished, take a look at the AWS S3 online, you will see a folder named 'static' created. All your static files are now stored in there.
7.
[Deploy to Heroku]
Quite a number of things are set. Now tell Heroku what we upgraded:
- You have installed 2 new packages, so you need to include them into your
requirements.txt
:
boto3==1.5.29
django-storages==1.6.5
- Also some AWS keys and ID entered in
.env
should be entered manually on Heroku dashboard:

- Change back to dj-database-url for using database on Heroku:
import dj_database_url
DATABASES = {
'default': dj_database_url.config(
default=config('DATABASE_URL')
)
}
"""
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'database_name',
'USER': 'postgres',
'DBPASSWORD': config('DBPASSWORD'),
'HOST': '127.0.0.1',
'PORT': '5432'
}
}
"""
Git add, commit and push the changes to Heroku. Now everything should be running smoothly with AWS S3! If you used
Whitenoise as static files adaptor previously, you may remove it and all related settings! As you are now using the S3!
p.s.
As you no longer use the static files in your project on Heroku platform but AWS S3, you may just simply disable collectstatic function (I don't recommend) by typing:
$heroku config:set DISABLE_COLLECTSTATIC=1
** DISABLE_COLLECTSTATIC=0 to enable it **
So that Heroku simply skips the procedures to collectstatic during deployment. Since you actually collectstatic from local command already, it really doesn't matter.