Thanks to very convenient timing of PyCon AU 2018 conference -- it is both after the NZIFF 2018 finished, and immediately before the AusNog 2018 conference being held nearby -- I have been able to schedule a trip to PyCon AU for the first time. (It has been on my "go some day" list for a few years.)

The conference consists of four "Specialist Tracks":

on the first day (Friday), then two days of the main conference, followed by development sprints on the Monday and Tuesday. Because I need to be in town later in the following week (for AusNog 2018), I even get to stay for the development sprints.

The conference is at (part of) the International Convention Centre next to Darling Harbour, which is a very pretty choice of location, although not entirely functional for a conference that encourages a lot of changing between rooms (one of the rooms is a non trivial distance from the rest...).

For the first day I spent my time bouncing between the Security and Privacy Track, the Internet of Things track, and DjangoCon AU. Below are some notes from the various talks that I saw.

Agloe - What the map makers of the 1930s can teach us about protecting our data in 2018

@errbufferoverfl started by talking about Agloe, New York which started its life as a "paper town", named after the initials of a set of map makers, placed on a map to help catch other map producers that had copied the original map. The highway intersection where it was existed, but the town did not... until a general store was built there, and then it ended up on more maps, in the county data, and even briefly on Google Maps. It was eventually too successful and became a psuedo real place, as the result of having been put "on the map", so no longer served its purpose as an indicator of copying the original map.

From there, the talk moved on to discuss IT based "fake information" to plant to catch people who are copying things, or poking around where they should not. This includes Thinkst Canary Tokens (on GitHub) and Atlassian SpaceCrab (on BitBucket), and other similar "boobytraps".

The talk was fairly short (at least compared with the time scheduled) but if you do not already know about Canary Tokens in particular it could be a gentle introduction.

Lighting Macro Photographs with CircuitPython

I missed this talk (it overlapped with the Agloe one), but did manage to get into the room in time to see a few of the devices shown off after the talk -- and they looked really useful.

Stacy Morse has done a bunch of work on long exposure macro photography, using LED lighting under programmatic control. The CircuitPython code is on GitHub. The hardware shown seemed interesting enough to make me want to watch the video of the talk, so I plan to do that as soon as I get a chance.

ETA 2018-08-29: the video is worth watching, particularly for the photos (in theory the slides are posted somewhere but I have not found them). There is also a blog post with details of the hardware used). The lights are NeoPixel Jewel 7-LED lights, with brightness by PWM (Pulse Width Modulation). These allow choice of light colour mix. Then a small microcontroller (running CircuitPython or MicroPython) can control the brightness, etc.

Writing Fast and Efficient MicroPython

When the creator of MicroPython, Damien George, talks about writing fast MicroPython it is well worth paying attention! He talked about both space efficiency (eg, RAM) and time efficiency (eg, CPU time consumed). Both matter for MicroPython as it is designed to run on small microcontrollers, which may have only 16KiB of RAM, and relatively slow (by modern standards) CPUs. The MicroPython Wiki page on speed contains many of these tips too.

For space efficiency he noted:

  • variable names are kept in RAM, so use fewer / shortered named variables

  • local variables turn into stack indexes, which are fast (and small)

  • global varibles and methods are a dictionary lookup, using a name, so need more space and take more time

  • many core constructs do not allocate memory, but beware that heap allocation happens on import, defining functions/classes, the first time globals are assigned (to have somewhere to store them!), creating data structures, etc. (Local variables are just put on pre-allocated stack space, so do not allocate memory.) The slides have a list of methods which do not allocate memory.

Other tips:

  • Cache method handles, and global variable handles, to avoid repeated lookups (he showed several variations, getting increasingly faster). This is a common compiler optimisation -- to hoisting loop invariants out of the loop, but it appears that MicroPython does not do this itself, at least for methods and global/class variables, so it can be a big win to do it "by hand".

  • Prefer a single longer expression over multiple partial calculations broken up, as the single expression can be done entirely on the stack without storage. (This is a form of Instruction Combining which is another common optimisation, that it appears MicroPython will not do on its own.)

  • Script mimification, using mpy-cross to precompile to byte code, or freezing code into flash can help with storage space.

  • Using the FOOinto() methods to avoid allocation (and read into an already allocated buffer, which may then only need to be allocated once). Memory allocation is relatively slow.

There are also some decorators specific to MicroPython which can allow for more efficient implementations for the specific functions which your profiling show are bottlenecks:

  • micropython.native causes native assembler to be emitted instead of bytecode that needs to be interpreted (like a "JIT", but done at compile time); the result is less flexible, but likely faster.

  • micropython.viper which allows writing assembler code in a Python-like syntax. It only works for relatively low level things (eg, register twiddling), but where it does work it can run a lot faster.

  • micropython.asm_thumb lets you write inline assembler in the native instructions of the underlying CPU (eg, ARM Thumb). The result is not portable, but can be as fast as hand written native code can be.

Damien showed several examples of blinking a LED, successively faster and faster -- starting at around 50kHz, and ending up at several MHz blink rate. So these hand optimisation steps can help a lot, especially for low level register/bit manipulation, if they are in the middle of tight loops. But profile first to find the hot spots before spending time writing unportable, difficult to maintain code.

Asyncio in (Micro)Python

Asyncio is a Python 3.5+ feature, which builds into core Python the concept of "green threads" and cooperative multitasking, using two new keywords: async def, and await. (The initial support was added in Python 3.4, but it is much more usable from Python 3.5 onwards.) It is also supported in recent MicroPython, with the same keywords, through uasyncio.

The basic ideas have been in Python for years in the form of Twisted, gevent and so on, but with it built into the Python language it is becoming easier to use, and rolled out through more code. Essentially at any point that a function might have blocked the CPU (or worse, spun CPU) it can instead yield the CPU (with await, or a method that calls await) and cooperatively let some other code make progress.

Matt Trentini illustrated how this is a natural fit for many MicroPython projects, which tend to be quite event driven and have one or more small tasks that need to run periodically but might be waiting on external events.

There is a good tutorial about using uasyncio on MicroPython for hardware interfaces written by Peter Hinch, which Matt recommended reading.

Learning from the Mistakes that Even Big Projects Make

Mikal Still, an OpenStack developer for many years, described the journey which OpenStack had been on in creating a more secure procedure for running privileged actions.

They started using sudo directly, leaving it "to the reader" to secure sudo (unsurprisingly hardly anyone could figure out precisely what needed to be allowed to be run through sudo...).

The next iteration used used rootwrap, to provide an OpenStack specific tool that could validate that the command being run was supposed to be run with privileges, and then that could be the only thing which needed to be allowed in the sudo configuration.

However there was still both a non-trivial overhead in running the commands (sudo, rootwrap, etc all needed to be run and parse their configs), and some things where much more difficult to express than being called directly (Mikal showed an example of abusing tee to set a value in a file and parsing the output to check if it worked... which is definitely obscure).

So their current solution is the oslo.privsep library, which transparently passes actions that need privileges off to a privileged process, and then returns the results -- the privileged helper process is started via sudo/rootwrap. There result is much faster, and more convenient for developers to use -- often just a decorator on the function that needs to run with additional privileges.

Mikal's talk (video), and the corresponding blog post contain a number of actionable items that could apply to other projects, including aiming for the path of least astomishment, and make it easy to use your API properly (and difficult to make use it incorrectly). He also suggested watching out for people cut'n'pasting bad examples that already exist in your code as they try to get things done quickly -- a few bad examples that do not hurt much can turn into many bad examples, and then become "the way" that something is "usually done".

Dynamic Web Pages Without Javascript

Spoiler: there is JavaScript -- but you do not have to write it!

Tim Bell gave a tutorial on using intercooler.js in your web frontend to automate dynamic updates in your web page. intercooler.js is configured with HTML attributes, such as ic-post-to, ic-target, ic-replace-target (a boolean), ic-trigger, ic-include, etc. When triggered it can POST a target URL, and then replace the entity (or a specified target entity) with the result of the POST call. This allows you to easily implement things like field validation as soon as the value is implemented, using the backend validation logic directly (although at the expense of a client-server round trip).

If you want that style of "on change" AJAX interaction with the server and do not want to write Javascript, it looks like a good solution that is easy to use piecemeal. Peter Curet has a blog post showing how to add auto complete without writing Javascript which uses intercooler.js, which was recommended by Tim as a guide for how easy it is to get started (although possibly not the most efficient way to implement auto complete).

Securing your Company's Data: Encryption, Deletion, and other best pratices

Elissa Shevinsky, an serial entrepreneur currently working with blockchain, was very pleased to have made it to Australia (from New York) -- and to be talking in Australia.

The talk covered the usual security and privacy topics:

  • minimise data collected

  • delete data

  • encrypt data at rest

  • limit access to data

and so on.

She pointed out that data can be valulable (eg, profiling users for targetting content at them), but it can also be a liability. The past few years have shown the liability of all the data collected, and then accidentally leaked/stolen/deliberately given away in ways that have created controversy. Elissa suggested considering whether you needed to collect specific data, especially personal data, and whether you need to keep it (eg, logs). Having a policy of not collecting, or deleting after a short period, greatly reduces the potential liability.

Elissa was a big fan of actually deleting data when the user requested it to be deleted, not just marking it as "deleted" and not showing it in the UI. There may be audit or legal reasons to need to retain the data in the database; but there are also legal implications in retaining data that the user has clearly indicated they wanted deleted especially in EU laws (GDPR and others).

She was also a fan of not storing passwords -- maybe use oAuth to another platform (eg, GitHub or Google) for authentication. But that of course comes with its own risk (eg, tying user identity together across multiple sites, which users may not want).

Elissa gave a shout out to the OWASP Python Security Project, which looks good, if possibly a little dated now (it is mostly aimed at Python 2.7, not Python 3).

Automating your Home with Python, Raspberry Pi, and HomeKit

Sean Johnson showed how to use the Apple HomeKit Accessory Protocol from Python with HAP-Python. HAP (The HomeKit Accessory Protocol) is what enables using the Apple "Home" application to control devices ("accessories") in your house, and HAP-Python allows you to implement custom accessories that the "Home" App can control, using Python. These can then be controlled directly when you are at home, or remotely if you set up a bridge on an AppleTV or an iPad that is left at home.

Sean's application was to control an IR-only air conditioner, using an IR sender connected to a Raspberry Pi controlled with LIRC. Conveniently lots of people have decoded lots of IR remote controls and implemented them as LIRC configuration files, so often you can just reuse the existing work. This mean that all Sean needed to do was write the HAP glue logic, using HAP-Python, to hook it all together.

If you are using the Apple/iOS ecosystem, and have other devices you would like to control, HAP-Python seems like a very useful solution to "write your own".

WebAuthn: Multi Factor Auth for Everyone

WebAuthn is an (almost final) W3C specification for authentication using per-user/per-site public keys stored on physical authenticator device, typically something like a YubiKey.

It allows a client -- typically a web browser -- to proxy a connection between a server (the "relying party") that wants to authenticate a user, and a non-network-connected device such as a hardware authentication token (via the Client to Authenticator Protocol -- CTAP. This permits per-user/per-origin public/private keys to be created at an enrollment phase, and then used later for authentication, without the authenticator device ever needing to talk directly to the server, or the private key ever being sent outside the authentication device.

It should be quite a useful protocol once the support is more widespread. Currently Chrome 67 and later and FireFox 60 and later support WebAuthn, with support coming to Microsoft Edge soon (with the release of version 18) and hopefully eventually to the Apple Safari/iOS platforms. (Chrome has also supported the older U2F protocol for a long time, and FireFox has some U2F support since FireFox 57 but it is disabled by default.)

Benno Rice suggested that websites using WebAuthn should allow users to register multiple authenticators (at least two), and let them add tags to the registrations so they know which device is which in case they need to deregister one of them (eg, it was lost or broken). He also suggested that if you need to lock down to a particular authenticator brand (eg, to ensure it is the official corporate approved one) you could check the root certificate of the public key presented.

Design for non-Designers

Tracy Osborn who has written a book which was kickstarted based on the blog posts inspired by the talk... gave an updated version of her "Design for non-Designers" talk, completing the circle.

Her key advice for beginner designers was "reduce clutter". The remainder of the talk is a series of practical tips on how to do that, including:

  • lay things out on a grid, so that the line up

  • keep colours complementary (opposite side of colour wheel)

  • use mostly neutral colours, with one colour for highlights throughout

  • use two fonts (eg, one for headings and one for body text)

  • use italics and bold if you need more variants

  • try to use fonts from the same family/foundary so that they are likely to be designed to look good together

  • avoid excessively "fancy" fonts, that can look busy/distracting

  • use more white space (maybe not the acres of white space of modern design, but give things room to breathe)

  • "less is more" for content; if you can express something concisely and directly, do that.

  • make the content easy to skim (eg, shorter paragraphs or bullet points :-) )

  • use succinct headlines (benefits, not details), with approachable human language

  • be wary of stock images -- they can look very "stock", and with modern displays often have to be large to look good quality. But some well chosen images may help.

She also pointed out many useful resources for getting ideas and inspiration, including:

Django against the Dark Arts

Lilly Ryan (@atticus_au) gave a fantastic talk analysing the security defences in Harry Potter, through three specific examples, complete with a robe, and wand.

You absolutely must see the video to appreciate just how awesome the talk was. It really was a fantastic way to finish the day.

From an actual practical Django security introduction she suggested James Bennett's Introduction to secure web development with Django from Pycon 2017 (a 3 hour tutorial). And some other talks mentioned in the slides.

(She also gave thanks to Podium, the Python/Markdown based slide tool she used.)