Schalk Neethling

Scripting on Caffiene

Configuring Django-Pipeline With SugarDough

| Comments

A few days ago I started a new project and decided to try out our (Mozilla Web-Dev) new Django web application template SugarDough.

SugarDough replaces our previous Django web app template Playdoh, and in general, starting a new project could not be simpler.

Currently however, a discussion is still in progress to decide whether to add something like django-pipeline or django-compressor to SugarDough by default so, for the time being, you need to add and configure this manually. This then, is where my jouney began.

Before we get to adding django-pipeline however, there are some additional steps required to get your project ready that is not currently mentioned in the README on Github. First ensure that you are in the root of your new project. You next step is to create a virtual environment for your project. If you do not already have a solution installed to do this, I would highly recommend Doug Heilmann's virtualenvwrapper for which you can find installation instruction here on ReadTheDocs.

Once this is installed, simply run the following in the root of your project.

Create a virtual environment
1
mkvirtualenv yourprojectname

Once the process completed, we can continue to install the required dependencies. Before we do that though, let's add django-pipeline to our requirements so this is installed right from the get go. Open the requirements.txt file and add the following lines:

Add pipeline dependency
1
2
3
# sha256: 4h6LijS01EbJtWCQ4yh847Y9GuHPbe8ejb-Ij0DkbMw
# sha256: Vibe-NP1i9L1ne67vn7-KckiAGpFFQblwH3VtKSQ_8M
django-pipeline==1.4.7

Feel free to use different versions than the one's specified above, just remember to update the SHA accordingly. Before we then install all of these dependencies, there is one slight detour we need to take to avoid installing dependencies we do not require. As you may have noticed, SugarDough is configured to use PostegreSQL out of the box. If this works for you, great! You can skip ahead. However, if you want to use a different database or no database at all, continue reading.

As you will see from the README, you can swap out PostgreSQL for another database by removing the psycopg2 requirement from requirements.txt and adding the relevant dependency, such as MySQL-python for example. If you are not going to use a database at all, simply remove the psycopg2 requirement.

If you are going to use Travis, you will need to edit the travis.yml file by removing or editing the following lines:

Edit travis config
1
2
3
4
addons:
  postgresql: "9.3"
before_script:
  - createdb tmp_db

If you will be using Docker, you will need to edit Dockerfile and docker-compose.yml to either remove the database requirements/config or substitute it with your chosen database. In Dockerfile, that would be:

Editing Dockerfile
1
apt-get install -y --no-install-recommends build-essential python-dev libpq-dev postgresql-client gettext && \

And in docker-compose.yml, that would be:

Editing docker-compose.yml
1
2
3
4
5
6
db:
  image: postgres:9.3
...
environment:
...
  - DATABASE_URL=postgres://postgres@db/postgres

Ok phew! Now we are ready to install those dependencies. In you terminal run:

Install requirements
1
pip install -r requirements.txt

Once all the dependencies have ben installed, we can move on to some config changes and, add django-pipeline. In projectname/projectname open the file settings.py. In INSTALLED_APPS add the following entry:

Edit settings.py
1
2
3
4
5
INSTALLED_APPS = [
    ....
    # pipeline for asset packing and less compilation
    'pipeline',
]

Now scroll down to the DATABASES entry. How you change this will depend on your needs. You can read more about database configuration in the Django docs but, even if you are not going to use a database for your project, Django does require some form of database back-end for it's own tables. If your needs does not go beyond this, you can use sqlite. For this, you need to change the DATABASES entry as follows:

Configure sqlite
1
2
3
4
5
6
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'yourproject_db',
    }
}

Ok, we are making progress. Next, scroll down to the STATICFILES_STORAGE entry and change it to the following:

Add pipeline storage
1
STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'

Now we need to tell Django which finders to use to locate our generated static assets. Add the following entries following the MEDIA_URL entry:

Add pipeline finders
1
2
3
4
5
6
7
8
9
STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'pipeline.finders.CachedFileFinder',
    'pipeline.finders.PipelineFinder',
)
STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'media'),
)

Next we need to add some django-pipeline configuration entries. Pipeline has a default configuration that will wrap all JavaScript in an anonymous, self executing function to prevent you from accidentally polluting the global namespace. If you follow best practices with regards to JavaScript coding and style, you will already know how to avoid this, and thus will not need this functionality. If this is the case, add the following entry to turn it of:

Disable JS wrapper
1
PIPELINE_DISABLE_WRAPPER = True

Pipeline also comes with support for precomilers such as SASS, CoffeeScript and LESS. In this post I will look at adding LESS support. Adding support for any of the compilers is real simple. To add LESS support simply add the following entry:

Adding LESS preprocessor
1
2
3
PIPELINE_COMPILERS = (
    'pipeline.compilers.less.LessCompiler',
)

Now, if you previously installed LESS globally using NPM, this is all you need but, if you are adding LESS using either NPM or Bower, you will need to add the following entry as well:

Configure LESS binary
1
2
PIPELINE_LESS_BINARY = config('PIPELINE_LESS_BINARY',
                              default=os.path.join(BASE_DIR, 'node_modules', 'less', 'bin', 'lessc'))

Pipeline bundles default support for the Yuglify compressor. I will look at the configuration for this in a separate post so for now, we will just disable any compression by adding:

Disable compressors
1
2
3
PIPELINE_CSS_COMPRESSOR = None

PIPELINE_JS_COMPRESSOR = None

One last thing left to configure and that is Jinja itself in the TEMPLATES configuration block. The defaults are pretty much what we want but, we need to add the pipeline extensions. In order to do that, we need to add a new extensions list. But here is the catch. If you add the following for example:

Add pipeline extensions to Jinja
1
2
3
"extensions": [
    'pipeline.jinja2.ext.PipelineExtension',
],

The pipeline extensions will be the only extension loaded into the Jinja environment and none of the defaults will be loaded. You don't want that, believe me. To remedy this, we add pipeline as well as all of the defaults.

Add default Jinja extensions
1
2
3
4
5
6
7
8
9
10
11
12
13
14
"extensions": [
    "jinja2.ext.do",
    "jinja2.ext.loopcontrols",
    "jinja2.ext.with_",
    "jinja2.ext.i18n",
    "jinja2.ext.autoescape",
    "django_jinja.builtins.extensions.CsrfExtension",
    "django_jinja.builtins.extensions.CacheExtension",
    "django_jinja.builtins.extensions.TimezoneExtension",
    "django_jinja.builtins.extensions.UrlsExtension",
    "django_jinja.builtins.extensions.StaticFilesExtension",
    "django_jinja.builtins.extensions.DjangoFiltersExtension",
    "pipeline.jinja2.ext.PipelineExtension",
],

Update: If you use django-pipeline v. 1.5.4+ you need to replace the last line from above with:

Pipeline Jinja extension in django-pipeline 1.5.4
1
2
3
4
"extensions": [
  ....
  "pipeline.templatetags.ext.PipelineExtension"
],

While we are at it, let's set a few additional configuration options. The first turns on autoescaping and the latter, turns on or off auto reloading dependent upon an environmental setting.

Additional Jinja config
1
2
"autoescape": True,
"auto_reload": DEBUG,

With that we are just about set. But wait, what was that about environmental configuration settings? Why yes, there is one more thing. In the root of you project create a new file named .env and to it, add the following entries:

.env configuation
1
2
3
4
DEBUG=True
TEMPLATE_DEBUG=True
SECRET_KEY=FOO
ALLOWED_HOSTS=localhost

As you can see, DEBUG is set to true here so, Jinja auto reloading will be turned on. If you now run:

Start development server
1
./manage.py runserver

Your development server should start up successfully. There is going to be one warning though, and we will look at this in a sec. For now, point your browser at localhost:8000 and you should see the "It works! Now build something" message. \o/

Before we move on and test that the LESS preprocessor actually works, that the static media is generated and loaded successfully, let's resolve that one outstanding issue. Kill the server with ctrl+c and run:

Apply database migrations
1
./manage.py migrate

Now start up your server again, and this time, it should be a nice clean start with no warnings. If you are not using a preprocessor, you are good to go. If you are however, keep reading.

Open up home.jinja in project/base/templates/project. Cut the CSS in-between the <style> tags, create a new folder in the root of your project called media, and inside that, a folder named css. Now inside the css folder, create a new file named main.less

Go ahead and paste the CSS you cut into this new file and save. Back in home.jinja, remove the <style> tags and in the head section of the document add the following:

1
2
3
{% block site_css %}
  {% stylesheet 'main' %}
{% endblock %}

Before we can start up the server and test this however, we need to tell pipeline where the files live that we want it to process. In the same folder where your settings.py file is located, create a new file called static_media.py and add the following to it.

Configure static media
1
2
3
4
5
6
7
8
9
10
11
# Static media bundle definitions.
PIPELINE_CSS = {
    'main': {
        'source_filenames': (
            'css/main.less',
        ),
        'output_filename': 'css/main-bundle.css',
    },
}

PIPELINE_JS = {}

Next, back in settings.py, add the following import following the decouple import.

Import static media config
1
from .static_media import PIPELINE_CSS, PIPELINE_JS

Great! Now pipeline knows about our bundles and you simply need to update this file when adding additional bundles and/or files. Alrighty! Go ahead and startup the server and reload the page. And that is that, now build something great :)

Comments