Ticketing is a hard problem. For the second year in a row NZIFF have elected to do their own ticketing, this time with veezi.com. And again ticketing has opened at 9am to high volume and site errors, followed a while later by requests for patience. Things did not really start working well until they increased their ticketing mojo in an emergency production outage.
Unlike last year, it took over 90 minutes of trying pretty regularly before I managed to buy my first tickets. In 2014 I eventually managed to get all my tickets in about an hour, after buying them in smaller chunks and just accepting any seats offered (they were mostly poor seats, but I did get to see the films). This year it took about an hour from being able to buy my first tickets (around 10:30) until I managed to buy the rest (around 11:30). Thank heaven the "wishlist" feature kept working, so I did not have to keep finding the sessions over and over again. On the plus side after the first couple of hours the load dropped enough that the seat selection mostly worked, and I managed to get reasonable seats for pretty much all my sessions.
This year there has been a wide variety of errors, including Varnish cache errors, "unexpected errors" adding tickets:
An unexpected error occurred adding tickets for ${MOVIE}. Please try again later.
extremely slow loads of pages, seat selection issues where the seats
could not be saved, sessions being cancelled and returned to the
"main site" (a https version of the main site, originally with a
fastly.net SSL certificate that did not include the nziff.co.nz
domain name).
But mostly after the first 45 minutes, it started showing:
Sorry!
Sorry, your request could not be processed right now. Click
the button below and try again.
Return To Homepage
any time you got close to purchasing any tickets, with some:
Error 503 backend read error
backend read error
Guru Mediation:
Details: cache-akl6421-AKL 1435704661 1364615349
Varnish cache server
errors thrown in for good measure.
At which point you had to start all over again, including trying to select tickets, and choose seats, etc. So definitely worse than last year. And unlike last year trying different browsers did not help at all, and trying to buy one movie at a time got tantelisingly close, but still failed. Overall it's not really obvious how many actual tickets they have sold, despite all the requests...
While pondering this problem I looked at the network traffic in Firefox. And was stunned to see that almost any page activity resulted in dozens of tiny requests (1-2KB) to the webserver. Literally even simple user clicks would result in 30-50 small requests to the webserver. No wonder it was melting under the load, and failing out regularly. Best practice for high load websites is to minimise the number of requests between the web browser and the web server, as it both adds latency (slowing everything down) and also increases the load on the web server significantly as most of those requests linearly increase the resources required by the webserver: if your page needs 30 requests, and you have 1,000 simultaneous users (tickets just went on sale!) then you are handling 30,000 requests "all at once". Even worse, it appears that most or all of these requests are uncachable, so going through to the backend servers, including for things like logos and images that should just be cached. That is presumably the cause of so many time outs and failures until they increased the backend capacity and the initial peak of orders went through.
Other problems with the ticketing website that impact performance:
the "best available seats" automatic selection algorithm is still poor, normally picking seats off to the left or right edge even when there are seats in the centre that are free. As a result I imagine a lot of people are trying to pick less bad seats, adding to the load.
the website delays asking for a lot of information, that will just need to be entered identically on each attempt, until quite late on. So that a lot of webserver requests and load potentially get wasted over and over again as users get thrown out at about 75% of the way through the purchase process.
I think that things would be dramatically better if users could:
add their desired seat to the wishlist, so that when they went to buy the system found the nearest seat to that which was still available; that may well avoid the entire tens of requests heavy "seat selection" GUI entirely, speeding users through the process
pre-enter all the information for their purchase (number/type of seats required, name, email address, physical address, phone number, etc), so that they were ready to go as soon as the correct sessions and seats were showing. (Pre-entering credit card information does not seem required, as the one part of the process that continues to work efficiently is Payment Express, who actually have designed their website to process requests efficiently under load.)
go directly from a failure back to the last step they were on, and try again, rather than being thrown all the way back to the top level of the site and having to re-add sessions and re-choose seats.
Plus of course it would help if NZIFF were not surprised by how popular their ticket sales were on the first day. After 30+ years of running film festivals, with routine queues to buy tickets on the first morning each year, it seems obvious to me that at least for day or two at the beginning, the website should be given much more resources to handle the early peak load. For all that "Cloud Computing" might be a terrible buzzword, it does at least provide solutions for having additional resources when extra demand is anticipated for a few hours or days.
All that said, once I got to the end of the process (about 3 hours all up: it is the only thing I have done this morning!), I was very pleased to see that I got pretty good seats this year. So I did send NZIFF a Kudos email for good seats, as it fixed my biggest complaint with last year (ie, poor seat selection).