Skip to main content

Simple self signed TLS server and client using Twisted

Self signed TLS server and client using Twisted

Prerequisites


Generate self signed certificate

Generate the server's private key using a secret, which is SuperSecretPassword in this case.
openssl genrsa -aes256 -passout pass:SuperSecretPassword -out server.key 2048
Perform a CSR (certificate signing request). Ensure the FQDN (fully qualified domain name) matches the hostname of the server, otherwise the server won't be properly validated.
openssl req -new -key server.key -passin pass:SuperSecretPassword -out server.csr
# Common Name (e.g. server FQDN or YOUR name) []:localhost
openssl x509 -req -passin pass:SuperSecretPassword -days 1024 -in server.csr -signkey server.key -out server.crt
For development purposes, remove the password from the certificate.
openssl rsa -in server.key -out server_no_pass.key -passin pass:SuperSecretPassword
mv server_no_pass.key server.key
Although not necessary, it's convenient to have the key and certificate in the same PEM file and the official Twisted TLS tutorial uses this method.
cat server.crt server.key > server.pem

TLS Server

A special object, called a contextFactory, is required for TLS connections. They can be generated in multiple ways. Generate a PrivateCertificate using a single PEM file. Notice the options() call, this produces a contextFactory object.
from twisted.internet import ssl

with open('./keys/server.pem') as f:
    certData = f.read()

certificate = ssl.PrivateCertificate.loadPEM(certData).options()
Alternatively, a contextFactory could be created using the certificate file and private key separately.
certificate = ssl.DefaultOpenSSLContextFactory('keys/server.key', 'keys/server.crt')
Finally, create and start a server.
from twisted.internet import endpoints, reactor, ssl
tls_server = endpoints.SSL4ServerEndpoint(reactor, 8000, certificate, interface='0.0.0.0')
# or
tls_server = endpoints.serverFromString(reactor, 'ssl:8000:interface=0.0.0.0:certKey=keys/server.crt:privateKey=keys/server.key')

tls_server.listen(my_factory)
reactor.start()

Example: TLS Server

import sys

from twisted.internet import endpoints, reactor, ssl
from twisted.web import server, resource
from twisted.python import log
from twisted.python.modules import getModule

class Example(resource.Resource):
    isLeaf = True

    def render_GET(self, request):
        return u'Hello World'.encode('ascii')

# create SSL server from string
https_server = endpoints.serverFromString(
    reactor,
    'ssl:8000:interface=0.0.0.0:certKey=keys/server.crt:privateKey=keys/server.key')

# start server
site = server.Site(Example())
https_server.listen(site)
log.startLogging(sys.stdout)
reactor.run()
The TLS site can be accessed at https://localhost:8000. If a web browser is used, then a warning prompt should appear to add the self signed certificate to a whitelist. Using curl:
curl --cacert keys/server.crt https://localhost:8000

TLS Client using treq

import treq
from twisted.internet import defer, ssl, task
from twisted.web import client

@task.react
@defer.inlineCallbacks
def custom_trust(_reactor):
    # get root cert from pem file
    with open('keys/server.crt') as cert_file:
        trust_root = yield ssl.Certificate.loadPEM(cert_file.read())

    # ready made browser-like policy
    policy = client.BrowserLikePolicyForHTTPS(trustRoot=trust_root)

    agent = client.Agent(_reactor, policy)
    treqish = treq.client.HTTPClient(agent)

    response = yield treqish.get('https://localhost:8000')
    content = yield response.content()
    print(content)
While most "requests-like" libraries allow ignoring of invalid certificates, treq and twisted do not. In conjunction with BrowserLikePolicyForHTTPS, treq can very easily handle self signed certificates by imitating how a browser handles such certificates.

Final Thoughts

Understanding and implementing a basic TLS server has been a very valuable experience. Some might be saying, "Why not use verify=False (requests) or -k (curl) to ignore certificates?" Although it's more convenient to ignore certificates when making HTTP requests, it's a very good habit to utilize TLS, even in development, and it's more secure. An unsafe side effect of ignoring certificates is that ALL certificates go unvalidated, which can lead users to unsafe sites.

Comments

Post a Comment

Popular posts from this blog

Python alias commands that play nice with virtualenv

There are plenty of predefined Python executables, symlinks, and aliases that come bundled with your operating system. These commands come in very handy because it saves you from typing out long commands or chain of scripts. However the downfall of operating system aliases is that they don’t always play nice with virtualenv (or venv if you’re on Python 3). Most predefined aliases use the system’s default Python as the interpreter, which is next to useless when your application runs in a virtual environment. Over the years, I’ve come up with my own Python aliases that play nice with virtual environments. For this post, I tried to stay as generic as possible such that any alias here can be used by every Pythonista. In other words, there will be no aliases for specific frameworks such as running a Django server or starting a Scrappy spider. The following is one of my bash scripts I source: .py-aliases #----- Pip -----# alias pip-list="pip freeze | less" alias pip-search...

Embed Scrapy in WSGI Application

WSGI and Scrapy A common question on Scrapy Stackoverflow is "How to use Scrapy with Flask, Django, or any other Python web framework?" Most are used to using the Scrapy’s generated projects and cli options, which make crawling a breeze, but are confused when trying to integrate Scrapy into a WSGI web framework. A common traceback encountered is ReactorNotRestartable , which stems from the underlaying Twisted framework. This occurs because, unlike asyncio or Tornado, Twisted’s eventloop/reactor cannot be restarted once stopped (the reason is a bit out of scope). So it becomes apparent that the trick to integrating Scrapy and WSGI frameworks involves being able to tame Twisted. Luckily, integrating async Twisted code with synchronous code has become quite easy and is only getting easier. In this post, the following will be demonstrated: Embed a crawler in a WSGI app and run it using Twisted’s twist web WSGI server. Embed a crawler in a WSGI app and run it any WSGI serve...