Fundamental Interconnectedness
This is the occasional blog of Ewen McNeill.
It is also available on LiveJournal as
ewen_mcneill, and
Dreamwidth as
ewen_mcneill_feed.
Crowdfunding, especially Kickstarter, has been one of the great successes of the Internet age, enabling creators to ensure they have a market before committing to the expensive parts of producing the product. It works best when the crowdfunding is aiming satisfying the first buyers of the project, with the up front crowdfunding payments covering the fixed costs needed to produce the project for those first buyers. Ideally the project is nearly ready for the (costly) final manufacturing steps at the point it is crowdfunded.
Feature films (both fiction and documentary) have also seen crowdfunding as a way to get indie film projects funded that they are unable to get funded any other way.
Unfortunately multiple aspects of the feature film industry make crowdfunding feature films problematic, and it seems to inevitably lead to considerably unhappiness on all sides. Having backed a bunch of feature film crowdfunding projects, and watched others from a distance, it rarely seems to work out well outside the ultra low budget "one person and a camera doing it all themselves, releasing only to the crowdfunding backers" area.
Some of the film industry specific problems with crowdfunding are:
feature films, even low budget indie feature films, are very expensive to make (usually a minimum of six to seven figures)
feature films are often crowdfunded at the "we have a script" stage, sometimes even before that (so funded very early on, not "just manufacturing left"), which make the budgets even less well known
it is very difficult to raise six or seven figures on a crowdfunding platform with little more than concepts to show for the project, so the feature film crowdfunding budget is normally set much lower than what is needed to finish the feature film
indie feature film crowdfunding campaigns rarely disclose that what is being raised by crowdfunding is (actually) "seed capital" for a project that may or may not be able to go ahead without substantial additional external funding (for fear of putting crowdfunding backers off, leading to even lower crowdfunding)
it is difficult to crowdfund a film without offering the backers a copy of the film when it is finished, or at minimum the chance to see it first before others
because of the high cost of making feature films, creators are almost always aiming at a much bigger market than they can sell to years in advance, such as film festivals, theatrical release, or mass market distribution. That broader distribution usually becomes the primary focus. This contrasts with most other crowdfunding where satisfying the buyers through crowdfunding is the primary goal, as those are most or all of the buyers (with the rare "breakout to mass market" exception for some very successful projects).
the main source of later stage feature film funding is selling the "nearly completed" project distribution rights (for theatrical or streaming distribution), normally only possible very late in the project when all but the final polish is done
feature film distributors traditionally want exclusivity, at least for the initial release period, which usually directly conflicts with earlier promises made to crowdfunding backers of downloadable copies "on release" or first to see via streaming
there are few other source of funding to finish the feature film, so indie feature film creators seem to inevitably sell out the promises made to their crowdfunding backers, leading to substantial delays in fulfilling the crowdfunding "rewards" (ie, up front promises). Even after the feature film is 100% finished and ready for release, there are usually additional delays before crowdfunding backers see anything. (Typically these delays are at least 1-2 years additional delay, on a project usually already finishing years after it was originally promised)
even indie feature films take a very long time to make (several years minimum) which is usually well beyond the delayed gratification timeline of even the most enthusiastic backers, so the promised timelines in the crowdfunding are inevitably the most optimistic possible timeline, and are typically missed by multiple years
there is also an awkward funding gap between the "seed funding" obtained by crowdfunding (with or without acknowledging that is what they are doing) and the distribution funding obtained at the end. Which can stall a project for many additional years, while the indie feature film creators struggle to find any additional funding to complete the next steps.
While it seems likely that most, if not all, indie feature film creators mean well when they run a crowdfunding campaign, the structure of the industry means that the timelines are missed by many years, and the crowdfunding backers -- almost by definition those most interested in seeing the feature film, willing to pay years in advance in the hope of seeing it first -- are prevented from seeing the film they funded with an implied "see it first", until well after the film is finished.
All of which inevitably means puts the crowdfunding creators and the crowdfunding backers into conflict, years after the project was funded, that only mounts up after the creator announces the film is finished but refuses -- often for years -- to let the backers see the finished film.
It's one thing to have a very delayed crowdfunding project -- that is common for anything except "order molds, manufacture, ship" projects -- but backers will understand while a creator is still working on the project.
However when a creator announces they project is "finished" and they have stopped working on it, but refuses to release the project or complete the backer rewards, that crowdfunding backer understanding of the delays fades away.
Those "post completion" delays feel unfair, because they directly contradict what was promised up front in order to secure the crowdfunding. And for some reason the creators typically seem very surprised by the reaction they get to announcing that "the project is finished, but you cannot have it" (in the way that was promised at the beginning).
Having backed a bunch of indie feature film crowdfunded projects, at this point I will not back any more feature film projects (except perhaps as a "no reward expected" small donation).
If you see an indie feature film crowdfunded project it is typically vastly better for everyone to just make a note of the title and check back in 5 years or so to see if it got made.
Most failed crowdfunding projects disappear, but the good ones manage to find other project funding and finish the film in a more traditional manner. They might return to crowdfunding at the end of the project to, eg, crowdfund producing physical media (eg, DVD / BluRay) -- at which point is is much more like successful crowdfunding as they just need to order production and the timelines are very predictable.
Feature film crowdfunding that caused conflict
Below are a couple of crowdfunding campaigns that I backed which ended up very delayed and either entirely refused to provide backer "rewards" or are still delaying providing any backer "rewards" 18+ months after finishing the project. And then in some later sections some that are still ongoing years late with patient backers waiting, or were more (eventually) successful in releasing the feature film to backers.
Blind Panic
Blind Panic is a feature length thriller based in Aotearoa by Mark Willis (director, co-writer) and Matthew Mawkes (producer, crowdfunder, co-writer). Mark Willis had directed one short film (and been assistant director on a couple more shorts) before starting Blind Panic (source: IMDB). Matthew Mawkes had produced one short film before starting on Blind Panic (source: IMDB). So as best I can tell Blind Panic was their first feature film.
Blind Panic was crowdfunded in 2016 on Kickstarter, just making its ultra low crowdfunding goal (NZ$40,000; so around US$20,000) after some in person appeals at the likes of the Wellington Film Society screenings (which is how I heard about the film). That NZ$40,000 budget turned out to be substantially too small, so the film completion was repeatedly delayed by years at a time trying to raise funding to complete the next steps (re-shoots, editing, colour grading, film score, etc).
Blind Panic was finally completed in 2025 (9 years later), and premiered at the Terror-Fi Film Festival in Wellington in October 2025. Sadly this was a few years after director/co-writer Mark Willis passed away (in May 2023, so Mark Willis never got to see the finished film get released.
The IMDB reviews, with a user review average of 5.8, feel about right -- it turned out a pretty good ultra low budget indie film, but it definitely shows that it is a low budget film. Worth seeing, especially if you are local to the Wellington region and likely to recognise some of the locations.
In March 2026, Blind Panic was released to streaming services, as its primary distribution.
Immediately prior to that, on 2026-03-02, the producer Matthew Mawkes directly emailed the Kickstarter backers offering a limited time (one week) streaming preview of the film (via Vimeo). That was the first opportunity any Kickstarter backers had to see the film, unless they were able to make it to one of the Terror-Fi Film Festival screenings (and pay again to see the film screened in a theatre).
In that 2026-03-02 email, Matthew Mawkes claimed "one of the rewards we promised you was the chance to see the finished movie upon release".
But that was not actually true.
None of the Kickstarter rewards promised "the chance to see the finished movie upon release".
Instead many of the rewards promised "THE COMPLETE MOVIE UPON RELEASE via digital download" (caps in original), and one very high tier reward promised "SEE THE MOVIE BEFORE ANYONE ELSE! You will get access to a special online screening of the film before the premiere" (ie well before "upon release").
In reality the time limited streaming access offered was just a (nice) bonus offering to backers, in advance of the actual Kickstarter rewards being delivered.
So I replied to that email, the same day, and pointed out that the backers were actually promised a "digital download", and asking if that would be available in 2026.
Matthew Mawkes replied back late that same day saying he was waiting for the film to be on streaming sites in Aotearoa:
[t]hen I can look into that digital download option for you. Just nervous about the download option at the moment with so many films ending up on torrent sites. But I will make a note to get in touch again when that is available.
Which seemed reasonable to me, as I was not very attached to the "at release" part of the promised reward, after nearly ten years. So I made myself a reminder to check in again in three months.
Having not heard anything further, three months later (June 2026) I replied to the email to follow up and ask about the timing of the digital downloads again. Matthew Mawkes replied late that same day with the surprising claim that:
As part of fulfilling the Kickstarter rewards, we provided all eligible backers with a free screening window to watch the complete film upon release. This fulfilled the “digital download” component of the reward, as the full movie was delivered digitally and made available to view in full.
And:
The backer rewards were completed with the screening access already provided.
Any reasonable definition of "digital download" involves the purchaser being able to... download the film, something which was disabled on the Vimeo time limited preview screening (and the Vimeo access password was changed to enforce the short time limit).
An extremely time limited preview screening also gives purchases no opportunity to rewatch the film later, which is one of the primary purposes of purchasing the download version of a film (to keep on your own computer) rather than simply watching once via one-off streaming access.
So it is not at all credible to claim that an online screening preview "fulfilled the 'digital download”" reward.
I pushed back firmly at that surprising new claim, pointing out that I was willing to wait until 2027, for the downloaded version, if the distribution agreements were preventing completing the download rewards in 2026 (something which had happened with other feature film crowdfunding).
Matthew Mawkes then switched to just repeatedly asking for my bank account information so he could refund me. After the third such request (one on 2026-06-08, and two on 2026-06-10) I gave in and sent him my bank account information.
Matthew Mawkes refunded the face value of my pledge (NZ$50), and marked the pledge as cancelled in Kickstarter.
I did not cancel the pledge myself, and actually pointed out I would prefer not to take more money out of the film's budget; instead Matthew Mawkes insisted on cancelling it, over my protests, when asked about the timing for providing backer rewards originally promised.
Due to inflation, NZ$50 in 2016 would need about NZ$67 in 2026 to have the same buying power, so effectively I paid about NZ$17 (in 2026 dollars) to get "behind the scenes updates" for 10 years, and a one-off streaming viewing of the film. Which feels like approximately what it was worth, but I would have felt a lot better about it if I had just found the film on streaming services in 2026 instead of experiencing the 10 years of delays, and a producer who refused to honour promises made in 2016 and then getting hostile about it.
It is obviously difficult to know whether Matthew Mawkes decided in 2016, when the film was crowdfunded, that he would refuse to make a download version available to backers (which would then arguably make it misleading advertising to promote the "COMPLETE MOVIE UPON RELEASE via digital download"), or whether Matthew Mawkes decided ten years later, in 2026, perhaps as a result of distribution agreements he decided to sign, that he was going to refuse to ever fulfill the "digital download" reward (which would confirm the unsuitability of feature films for traditional crowdfunding).
But either way it turned what could have been a "successful crowdfunding, eventually" story into a failure case :-/
Ironically, as of mid June 2026 it seems like Blind Panic is no longer available on movie streaming services (or anywhere else as far as I can tell). Even the official website just says "coming to streaming services soon", and the streaming sites that still list it indicate the streaming rights have expired. Despite the fact that the film was on streaming services just a few months ago, as the official distribution.
Also ironically, the "OFFICIAL THANK YOU" part of the reward, which was not in the film credits and only on the website (eg, in 2017) was removed from the website well before the movie was released. (Last archived version of the "thank-you" page is 2019, about 6 years before the release of the film.) And as of the point when I was ejected from the Kickstarter, by the producer Matthew Mawkes, "THE FILM'S ORIGINAL SCORE" reward had also not been delivered to backers (ie, 8 months after the film premiere).
Matthew Mawkes noted that "it is a miracle that the movie was finished at all", which I would agree with.
The budget raised (NZ$40000) was massively too small to complete the project they seemingly actually wanted to complete (which I would guess needed NZ$150,000-NZ$200,000 to finish, and would still have been an ultra low budget film).
But the Risks and Challenges section of the Kickstarter gave no indication of just how unrealistically low the budget was to complete what was promised. Or just how unrealistic the "estimated delivery" of "Sep 2017" (1 year!) was to complete the feature film they actually had in mind (which in reality took took 9 years to complete).
A 1 year timeframe, and a NZ$40,000 budget, for a feature film implies something like a "grown up" version of the "48 hour film festival" shorts, entirely DIY in a compressed time frame, and done in an ultra-low-budget style.
In reality the NZ$40,000 raised by crowdfunding functioned much more like seed funding for a larger project that Mark Willis and Matthew Mawkes apparently wanted to complete, with the final funding gaps closed by distribution rights agreements that apparently blocked fulfilling the originally promised backer rewards. The results I think were probably worth the money spent, but I have a feeling the project is unlikely to break even, let alone make a profit. And the crowdfunding backers got only a small subset of the promised rewards, even after the film was eventually finished.
Arcade Dreams
Arcade Dreams is a three episode (later five episode) documentary series about arcade games from pinball and other mechanical games through the video arcades of the 1980s and beyond. It was crowdfunded by director Zachary Weddington in late 2020, raising a little over its goal of around US$100,000. Zachary Weddington had made one previous feature length documentary, also crowdfunded (and apparently followed a similar "much delayed" pattern to Arcade Dreams).
Estimated delivery was mid 2022, roughly 18 months after the crowdfunding closed. As of mid 2026, the documentary series has not been released to backers, nor has it seen any general release.
Production of the documentary series was basically done by late 2023, and in August 2024 they were telling backers they were "still planning to release Arcade Dreams this year". By September 2024 they had a formal offer from a distribution company, and it seemed quite promising but did not turn into an actual release in 2024.
Throughout 2025 (eg, March 2025, June 2025, September 2025, October 2025) Zachary Weddington gave the impression the documentary series would be released in 2025. Including in October 2025 saying "Arcade Dreams will be released by the end of the year at the latest".
None of that happened, and project updates on Kickstarter went silent from October 2025 until May 2026 when Zachary Weddington told backers "We are still moving forward" and that he hoped to hear from the distributors in May 2026. Since there has been no update for over 2 weeks since May 2026 ended, I assume he did not hear back from the distributors.
So essentially the project was mostly finished two and a half years ago (give or take some final polish and legal clearances), then got sold to a distributor who among other things bought the right to determine when it got released. And as a result, nothing has been released, there is no known timeline for a release, and backers have only seen short teaser snippets (a few minutes here and there) of the finished project.
Like Blind Panic above, the US$100,000 budget was clearly too small for what Zachary Weddington ended up wanting to make, so other funds were raised in larger chunks from independent backers, and then (I assume) yet more from the distributor to enable finishing the project. At a guess the total budget was more like US$250,000 to US$300,000 (still a fairly "indie production", but well above what was crowdfunded). That extra money came with restrictions which meant the original backers have yet to see much for their money put down 6 years ago.
To their credit, the Kickstarter Risks and Challenges section does say:
The number one risk for any project like this is going over-schedule or over-budget.
But they immediately followed that by saying:
We’ve structured the project with a realistic budget and time-frame based on the work we’ve already done on this project and comparable projects in the past.
none of which actually seems to be true in hindsight. The money and time budget really required seems to be at least 3x higher than what was originally allowed. And the project clearly expanded further in scope during the production process.
It remains unclear if the (basically finished apart from final clearances) documentary series will ever get released, and even if it does it will be at minimum of four years after the indicative release timeline and a couple of years after backers were told the documentary series was finished.
Understandably several of the crowdfunding backers are frustrated at being told the documentary was finished 18+ months ago, and yet still none of the backer rewards have been fulfilled. And it seems like Zachary Weddington surrendered control over the timing of releasing anything to crowdfunding backers to get additional (distribution) funding :-/
(Arcade Dreams by Zachary Weddington should not be confused with the Jason Scott planned documentary of the same name from about a decade earlier; Jason Scott passed on the documentary name to Zachary Weddington after Jason Scott decided not to go ahead with making a documentary about gaming arcades, due to day job time commitments.)
Very delayed, over budget, but still working on the project
Recently Deceased: The Making of Beetlejuice
"Recently Deceased: The Making of Beetlejuice" is a fan made "making of" documentary, that was Kickstarted by Fred China in March 2019, raising about 50% more than its ultralow budget target of EUR 20,000 (just over EUR 30,000 raised). Even the higher amount raised really was not enough for the scope of the project (especially at the historical distance it was being made), and unsurprisingly the project has proved rather large to be a single person project.
The original release target of December 2020 (so about 22 months from funding) was fairly optimistic even before the Covid-19 Pandemic restrictions interfered with everything in the world.
To his credit Fred China has kept working away on the project, posting periodic updates, and as of May 2026 is nearly at a point of completing the planned "making of" documentary. Unfortunately due to spending "nights and weekends", as well as periods of full time, on the project it sounds like it is well over budget for the EUR 30,000 raised, even without considering things like producing and shipping physical media as was originally planned.
I am hopeful that the project will get declared "finished" and released to backers, possibly even in 2026.
But the most recent update (May 2026) concludes with the comment that "[h]opefully the next newsletter would be about a future deal with a distributor", so I suspect this may be another project that closes the funding gap by selling the project to a distributor... who then exercises control over when it is released, and when the original backers even get to see the documentary they funded 7+ years ago.
The Myst Documentary
"The Myst Documentary" is also basically a fan made "making of" about the Myst game series, funded on Kickstarter in July/August 2020, raising about US$260,000 (somewhat above the funding target of US$200,000).
Philip Shane, the Director and Kickstarter Creator, does have a fairly extensive history in documentary film making, and is a fan of the games, so even though it was originally funded as a "low budget indie" film it is also very much a passion project, which hopefully will mean the documentary does (eventually) get finished.
The original aim was to complete the project by December 2022 (and they were funding in the midst of the initial parts of the Covid-19 pandemic so at least knew what they were getting themselves into with regard to schedule disruptions).
However the project used up all of its budget by about 2023/2024, without having a completed project to show for it, and has been hunting for substantial additional funding ever since. Work has gradually continued on the project as a backburner item, including starting editing the already shot footage together. And there have been annual updates at the Mysterium Convention showing a bit more progress each time.
To their credit the Risks and Challenges section of the Kickstarter does say:
It’s possible that additional time and/or funding would be needed to complete the film in its later stages.
but for an experienced film making that feels fairly soft pedalling what seems, in hindsight, to be the near certainty of needing to raise substantial additional funding to complete the project.
I am fairly convinced that Philip Shane wants to finish the documentary eventually (since it comes across as a passion project), but it seems likely to take at least a few more years. Hopefully by 2028 or 2030 or 2032 it might be finished.
Unfortunately it also seems extremely likely that it will eventually get finished enough to sell into distribution, coming with its own restrictions on backers being able to see the finished project. (Which is unfortunate as the approximately 3000 backers of the Kickstarter funding are probably the exact people in the world who most want to see the documentary, and likely to end up being the last to be able to see it -- at least without it appearing in a Film Festival or streaming service near them and paying again to watch the documentary.)
Completed, backers got their rewards well after the film was completed
MAKING WAVES: The Art of Cinematic Sound
"MAKING WAVES: The Art of Cinematic Sound" is a documentary about the place of sound in cinema. It was crowdfunded in mid 2017 by "Ain't Heard Nothin' Yet Corp.", a group of film lecturers and professors, and someone from industry.
The Kickstarter raised about twice their goal -- about US$135,000 raised, on a goal of just over US$70,000. And for once that budget, plus whatever they raised through distribution, seems to have been sufficient.
The original plan was to provide backer rewards around August 2018 (so about 1 year after funding), but backer rewards did not actually ship until January 2020. However backers did get to see an online streaming preview a few months earlier in October 2019.
Unfortunately those backer rewards, including the online streaming, came after the documentary had been through the Film Festival circuit and plenty of other people had seen the film. That was one of the few crowdfunded films that I paid again to see in the 2019 NZIFF Film Festival, months before the Kickstarter backers got online streaming access. It was somewhat disappointing to have paid for what I assumed at the time would be to be one of the first to see the documentary, only to find out that they were planning to leave fulfilling backer rewards until after the film was fairly widely released.
I guess the Film Festival screenings probably considerably helped the documentary's budget. But the choice to do that before providing anything to the backers who funded the film, and delay the backer rewards by 12-18 months, illustrates one of the disadvantages of the feature film industry for crowdfunding backers.
The plan to use film festival screenings in advance of (late) backer rewards was not disclosed up front. Indeed the Risks and Challenges section of the Kickstarter page says:
In summary, we say with total confidence - when the Kickstarter campaign succeeds, the film will be made; and all rewards will be delivered fully and on a timely basis.
which was an excess of confidence, as the documentary was not even finished until a year after the planned backer reward date, and not delivered to backers until several months after that completion.
I suspect the (first time) creators were just over optimistic in their ability, and did not take into account the reality of film distribution processes impacting fulfilling backer rewards.
But that experience (of having to pay again to see a film I had already paid for) was part of what started making me wary of feature film crowdfunding in general.
Keeper of Time
"Keeper of Time" is a feature length documentary about mechanical time keeping, especially the mechanical watch making. It was crowdfunded in late 2018 raising a little over its US$100,000 budget.
Backers were promised rewards in late 2019 (so a little over a year after crowdfunding), which proved to be quite overoptimistic.
The documentary eventually premiered in April 2022 to an in-person New York audience, although it had been finished quite some time before that (around May 2021). I assume that the distribution process again delayed the release of the documentary, in this case by about a year.
To their credit, the creators provided backers with a download (from Vimeo) at the same time that the film premiered. Although it was a very short duration download -- 24-48 hours only -- after which backers were instead given a code for unlimited streaming, also from Vimeo. (This contrasts with Blind Panic which only provided backers with a short duration streaming link, instead of even a permanent access streaming link, and waited until 6 months after the premiere to do so.)
Feature film crowdfunding success stories
When we were Apollo
"When we were Apollo" is a feature documentary about the Apollo program, originally crowdfunded in late 2017, and then followed by a second crowdfunding campaign a year later in 2018 to be able to complete the project. Between the two crowdfunding campaigns it raised about US$55,000, entirely from crowdfunding.
The original goal of backer rewards by February 2019 (to be early for the Apollo 50th Anniversary) slipped a bit, but backers had a downloadable version (a true "digital download"!) to watch by July 2019 and the physical media rewards followed (from memory) late in 2019.
Other than the need for a second Kickstarter to get enough funds to finish the project (which honestly felt like a good way to handle the short fall, and raised about 75% again over the first Kickstarter), the project seemed a very successful Kickstarter.
Zachary Weil, the creator and director, went on to make another space-related documentary for PBS (funded by PBS) -- When We Were Shuttle, and started a third space-related documentary in 2025 ("Erased for Space").
While Zachary Weil has since scaled back that third film project ("Erased for Space") to a YouTube series, he was clear in the Kickstarter "Budget and Timeline" section that the funding was clear that it was to "support an initial round of production", which did happen, rather than specifically to release the documentary to completion.
I suspect that "initial round of production" caveat, in the middle of the long crowdfunding description, took some backers by surprise, but given the honesty in the previous Kickstarters and producing "two documentaries for the price of one" I was at least willing to back the third Kickstarter at a "donation to get you started" level ("digital deluxe") without expecting to see anything more from the crowdfunding. So any "Erased for Space" parts released to YouTube will be a bonus.
First to the Moon: The Story of Apollo
"First to the Moon" is a also a feature documentary about the Apollo programme, specifically about Apollo 8 which was the first mission to fly around the moon.
The documentary had an earlier unsuccessful Kickstarter, which got to a similar US$65,000 amount but failed because the original goal was US$100,000; and then a second Kickstarter funding round successfully raised a similar approximately US$65,000 for the project (on a lower US$50,000 goal). That is a fairly clear indication of the difficulty of raising a realistic feature film budget via crowdfunding -- the crowdfunding only succeeded by halving the budget from "low" to "very low".
Backer rewards were aiming for February 2019 (also to meet the Apollo 50th anniversary), and did ship around June 2019 so relatively on time. Looking back at the backer updates it appears that there was a film distribution deal that delayed the release slightly (from May 2019 to June 2019), but compared with many other crowdfunded films that was a very minor delay due to film distribution.
Overall the documentary was completed, pretty much on time, and the backers got their rewards also pretty much on time, so I have quite positive memories of this crowdfunding project. And it stands out as somewhat unique for that reason.
Heart of Neon
"Heart of Neon" is a feature documentary about Jeff Minter and his very colourful games.
There was an initial attempt to crowdfund it in late 2019 which came up well short of its goal (getting about US$27,000 in pledges on a goal of around US$100,000).
To his credit, Paul Docherty, director and crowdfunder did not give up, and:
separately raised the money to be able to make the documentary; and
once the documentary was finished came back to Kickstarter in 2025 for funding to produce physical media for the documentary
The aim of the second Kickstarter was to ship physical media in August 2025, which slipped a bit, as projects do. But the download version was available to backers in October 2025, and the physical media shipped to backers around March 2026.
While it is unfortunate that crowdfunding did not manage to fund this documentary up front, it does demonstrate that arriving at Kickstarter with a completed documentary that just needs to be released greatly speeds up the process from the perspective of the crowdfunding backer, and dramatically reduces the risk to the backer of not getting the rewards.
Since I backed the original 2019 campaign, I can tell it still took about 5-6 years to make the documentary (with independent funding). But by the time the physical media part was successfully crowdfunded it was just a manufacturing problem, and not a "delays of making a feature film" problem.
Mixed results
8-bit Generation
This was intended to be a trilogy of documentaries, by the same creators, but it did not work out that way.
The story is also murkier because my understanding is that this was the second attempt by these creators to crowdfund the first two documentaries, and the first attempt (to crowdfund the first two documentaries together) did not result in anything getting to backers. (Apparently they eventually refunded some backers, who asked for refunds.)
Growing the 8-bit Generation
"Growing the 8-bit Generation" is a feature documentary about the early home computers, which was crowdfunded in mid 2015. It raised about EUR 40,000 (about 50% higher than its EUR 25,000 budget).
The aim was to complete the project by about February 2016 (which seems unrealistically short now I look at it -- about 7-8 months!), and they did manage to have streaming-only versions available to backers by about March 2016 which is fairly impressive in hitting timing projections. However the final backer physical rewards shipped about a year late, around mid 2017.
So basically a fairly successful project, although possibly helped by being (to my understanding) the second attempt to complete the documentary.
Easy to Learn, Hard to Master -- The Fate of Atari
"Easy to Learn, Hard to Master -- The Fate of Atari" is a feature documentary about the Atari company. It was crowdfunded in mid 2016, shortly after the first documentary had been released, and raised about EUR 25,000.
The aim was to complete the project by about February 2017 (again a fairly short turn around), and backers did get to see an online version of the documentary on time, around February 2017. Physical media was shipped in the second half of 2017, which is also fairly close to the intended timeline.
So also basically a fairly successful project.
Firing Steve Jobs
Having produced two documentaries with footage they already had they then went on to try to crowdfund a third documentary based on other parts of that existing footage, capitialising on the fact that Steve Jobs had recently passed away.
The crowdfunding on Kickstarter in mid 2017 did not get enough momentum and they cancelled the Kickstarter at about 20% of their target.
Then they switched to Indiegogo, to try again with a EUR 0 minimum, and raised about half the amount of the Kickstarter and 10% of their original budget.
The claimed in March 2018 that:
we are thrilled to publicly announce that also this third documentary will be available and finished this fall
and:
Despite will not probably meet the public goal we have reached an external agreement that gives us the opportunity to work and finish also this documentary.
but absolutely nothing has ever been produced from that crowdfunding campaign.
Eventually a few years later (in 2021) a backer noted that they had announced on their Facebook page that:
Quick news worth sharing: please note that one of the two creators, Tomaso Walliser, died few weeks ago.
In practice I think that 2021 passing almost certainly means that nothing will be produced, which is something of a sad end to a trilogy of three documentaries.
That experience had made me particularly wary of projects which fail on Kickstarter and then switch to Indiegogo "flexible funding" (ie, take all your money even if it is not enough to do the project). Indiegogo by itself is an okay funding platform, but switching from a funding failure to "flexible funding" is in hindsight a huge warning sign that the project will not be completed and the money given in funding will just disappear.
I do not know if the creators intended to ever actually make the third documentary, but the fact that they never posted any updates to the crowdfunding campaign after it was funded, for 8+ years, seems to at least indicate they lost interest and were not inclined to try to complete anything, or refund anyone.
Unfortunately of the three "8-Bit Generation" films, the third one was my largest contribution, and I just had to entirely write that off as a learning experience.
Conclusion
There are a few other feature films that I helped crowdfund, but the above is more than enough to give a flavour for the problems of crowdfunding feature films.
Even where the crowdfunding is successful, the feature films are often at much much earlier stage when they are crowdfunded than many other projects (books, music, physical items) which dramatically increases the chances of going over budget (time and money) in ways the creators do not anticipate and do not warn backers about up front.
Then the film industry distribution tends to take over and start dictating release processes and schedules, usually completely ignoring the backers who provided the original funding for the project so that the original backers end up feeling like an afterthought.
Often this ends up with the creators simultaneously announcing "the project is finished" and also "you cannot have it", which as a backer feels extremely unfair. And a very poor way to treat the very people who most wanted to see the film you are producing.
In most cases I do not think the creators set out to create this unfair situation, but their actions of prioritising their own desires (eg, mass distribution of their film, additional income) over the promises they made -- often years in the past -- inherently creates that resentment. And few creators seem to even acknowledge that they caused that resentment through their actions, even if it might not have been their primary intention.
I have a Brother HL-2270DW laser printer, that I bought in 2014, which is still going fine almost 10 years later. As the article says "buy the Brother laser printer, it's fine".
The Brother HL-2270DW supports (wired) network printing and WiFi
printing, but unfortunately its design pre-dated Apple's creation
of AirPrint and the
printer industry creation of IPP
Everywhere (both AirPrint
and IPP Everywhere were created around 2010, but took a while to
become common in printer designs). Which means (a) it needs a
printer driver (brlaser), and (b) while printing from Linux and
macOS works fine, printing from a tablet/phone did not work.
Mostly I just ignored the lack of printing from a tablet/phone as I did not have any reason to do it very often, and for those cases I could always re-open the link/document on my desktop to print it.
However a question on the fediverse, about relaying AirPrint to an older printer, got me investigating AirPrint support for older printers again, and I discovered that someone else had got CUPS working to provide an AirPrint wrapper to my exact printer. So I wanted to give it a try.
The process of getting CUPS working as an AirPrint relay seems to have changed a bit since CUPS 2.4.0 so while the guides I found online helped a lot to point me in the right direction, I had to do some debugging after the "and now it should just work" statments in others guides. The Debian CUPSAirPrint page had lots of useful information on how it was supposed to work, but also described an slightly older setup (around Debian Buster/10; whereas I have Debian Bookworm/12).
With Debian Bookworm (aka Debian Linux 12), the CUPS version provided is 2.4.2. And it seems like there are four key steps to getting CUPS working as an AirPrint relay:
Install
cupsand theprinter-driver-brlaser(CUPS driver for the Brother laser printers)Make sure that the printer is set up in CUPS as an
ippprinter (or maybeippsordnssd;ippwith a static IP on the printer worked for me, and Dustin)Ensure that the printer is shared in its printer setup
Ensure that printer sharing is enabled in CUPS (ie, system wide configuration to enable the printer sharing features of the printers that should be shared)
With all of those steps, then the CUPS daemon seems to be responsible
for advertising the printer into
mDNS /
DNS-SD
(aka Avahi, Bonjour, etc). Which is unlike earlier versions where
there was glue logic in the Debian avahi which did this
readvertisement.
So, after all the dead ends were eliminated, my process was to
install CUPS and the drivers for the Brother HL-2270DW laser printer
(brlaser):
sudo apt-get install cups printer-driver-brlaser
then set up the printer for local printing from Linux. In my case I did this via the CUPS web admin interface, as this Debian Linux system was a home server. The web admin interface is available at:
http://localhost:631/
and by default will authenticate with a Unix user/password (possibly in some sudo/admin group) for admin tasks.
Once the printer-driver-brlaser package was installed, there was
a Brother section to the printer vendor list, and the HL-2270DW
was explicitly listed. Then I could choose default options for the
printer, like defaulting to double sided printing. (The printer
showed up via DNS-SD in the list of network printers to set up,
which I did iniitally, but I ended up explicitly setting it up as
an ipp://DNSNAME/ipp/port1 printer when I initially had trouble
betting it to show up in AirPrint, and never went back and tried
the other options again.)
While setting up the printer (or if you missed it, while "modifying" the printer setup), you can choose to "Share this printer", and you do wnat to choose that for the Brother HL-2270DW that you want to re-share back onto the network.
The final key step to get the AirPrint advertisements from CUPS working is to go to the central admin page of CUPS:
http://localhost:631/admin
and make sure the "Share printers connected to this system" option on the right hand side is ticked. It seems to default to not ticked, which means no printer sharing happens at all (even if you indicated that a specific printer should be shared at the printer level). When you select "Share printers connected to this system" and click on "Change settings" the main CUPS configuration file is rewritten with that change, and the CUPS daemon is restarted.
After those steps, especially the final system wide enabling of CUPS sharing printers, I could see the printer available for AirPrint on my iPad.
The Debian CUPSAirPrint page contains some very useful advice on debugging this setup. In particular:
sudo apt-get install avahi-utils
will install the command line tools for examining what is advertised into mDNS / DNS-SD, and is invaluable for seeing whether the right announcements are being made for AirPrint to work.
Specifically you want to run:
avahi-browse -rt _ipp._tcp
and make sure there is an advertisement that includes image/urf
(the Apple Raster
Format).
If you are not seeing an advertisment with image/urf, then the CUPS
printer sharing is not working. In particular you are probably only
seeing the Brother HL-2270DW native advertisement of its capabilities,
which does not include image/urf (and hence AirPrint does not work).
While debugging this I ended up running;
avahi-browse -rt _ipp._tcp | grep image/urf
repeatedly, and as soon as I could see an advertisement with image/urf
then AirPrint worked (and AirPrint did not work at all before then).
(Usually there will be multiple advertisements: one set from both
the printer itself and another set from the Debian Linux system with
CUPS running. These cover the various service features of the printer.)
You can also install avahi-discover if you have a GUI Linux system,
which will provide a graphical view of the same information as avahi-browse
finds; but I found it more useful to be able to grep the output of
the command line tool.
Unfortuately CUPS is planning to deprecate printer
drivers entirely,
although what to do about some older
printers
still seems to be an open question. Which means using CUPS as an
AirPrint relay will probably break in a future upgrade :-/ But
it might at least be possible to keep an older Linux/CUPS install
running as the AirPrint bridge at that point. Or maybe there will
be some other application packaging the brlaser driver into an
"IPP Everywhere" printer front end, that CUPS will be willing to
talk to. (From the DNS-SD advertisements, it also looks like this
2024 CUPS install might be able to act as an IPP Everywhere relay
too. So that may be sufficient for the next few years.)
Introduction
The (original) Linux i386 architecture is pretty close to being removed, especially from Debian Linux (but also from the Linux kernel). The last few years have brought several announcements of "this is the last release with i386", only for there to be one more "last release" with i386 as a supported architecture. And various third party software has dropped support for i386, or is in the process of doing so.
Most relevant to me, SaltStack announced a switch to "onedir" packaging about a year ago, from Salt 3005 and especially Salt 3006. The "onedir" packaging means that Salt becomes a binary distribution, bundling its own Python interpreter -- which is obviously architecture specific. This means that my work around of using the "amd64" Salt packages (for older Salt releases) with an i386 Python (and some helper back ported Python modules) will no longer work. So that prompted me to finally convert the last few i386 VMs that were managed by Salt across to amd64 VMs.
The i386 cross grade approach I followed was fairly similar to my
earlier test Debian i386 to amd64
crossgrade, but
required some tweaks to get it working for Debian Bullseye. Especially,
as the Debian CrossGrading
Wiki page points out, Debian Bullseye requires perl-base cross
graded even earlier in the process, before the rest of the perl
packages. And there are some twisty perl / Debian packaging tool
dependencies to resolve early on.
This post documents the approach I used on three Debian Bullseye i386 systems to cross grade them to Debian Bullseye amd64. (At the time of this writing Debian Bookworm is the stable Debian Linux; but I decided to do the cross grade on these system before upgrading them to Debian Bookworm, as the Debian Bullseye cross grade challenges were better documented.)
If you are considering attempting this cross grade approach yourself:
know that you will end up with a broken system in the middle of the cross grade, and it will take care and experience to move forward into a working system again. Key packages you rely on might be auto-removed, or require data or configuration to be recreated to continue working.
make backups before you start. More than one. Be very sure you can restore from backups. If you are cross grading a VM, consider if you can minimise other writes to disk during the cross grade (eg, firewall the VM off from incoming network connections) and snapshot the VM disk image before you start, so you have an undo option of rolling back to the snapshot and pretending you did not try the risky crossgrade. But either way make sure you have a backup handy you can look at to see "how it was before" to guide you in putting the system back together.
especially if it is a VM, consider making a clone of the system in a virtual machine, and doing a few dry runs of the upgrade process in a VM you are happy to throw away. That should let you encounter most (but maybe not all) of the problems in a safer environment where you can roll back and try again.
at least twice during the cross grade you will have to tell the Debian packaging tools that you know what you are asking is a terrible idea, but you want to do it anyway (
Yes, do as I say!literally entered). Neither Debian nor I take any responsibility for your broken system as a result of attempting to cross grade it from i386 to amd64 following this or any other guide. You need to assure yourself that you have the experience to carry out this complicated risky procedure through to a safe conclusion.be aware that some packages (especially databases) have architecture dependent file formats on disk, which will break horribly if you change architecture. Make sure you have a plan to recover a working system from that cross grade (eg, in the case of databases make sure you know how to dump and reload the database before/after the crossgrade).
My first few trial attempts (in a VM clone) of the cross grade with Debian Bullseye took 1-2 hours to figure out all the required steps. By the time I had reduced each VM clone upgrade to a runsheet of steps specific to that VM install, the production crossgrade took about 30 minutes. But it took at least half a dozen attempts to get a "do these things, in this order" run sheet, that I was happy to attempt in production. Per VM.
If you have some other way to reinstall the system (especially automatically) on a new VM that is 64-bit from the start, that would be a safer option. This risky crossgrade procedure is probably only relevant to long standing pets which are difficult to replace. (All of mine were originally installed 15+ years ago, and have been through many Debian hand upgrades of Debian Linux releases since the install; two of them had been through a physical to virtual conversion too.)
Crossgrade preparation
Make sure your system is capable of running 64-bit code (especially if
it is a VM, make sure the VM "hardware" is ready for 64-bit code). The
lm flag ("Long Mode") needs to be visible in the CPU flags, eg:
ewen@debian:~$ grep ' lm ' /proc/cpuinfo
flags : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm cpuid_fault pti
ewen@debian:~$
Or check with:
lscpu | grep 64-bit
if you have lscpu installed.
Make sure you have access to the system console (eg, via a serial console
or a virtual machine console, or a remote KVM switch -- or maybe by
being local to the system). At one point in the upgrade you need to
select a specific kernel version, from the grub boot menu, which requires
console access.
Finish any package installs, and upgrades on the Debian Linux i386 system,
and make sure any earlier upgrades have been tidied up. Eg, dpkg --audit
should be clean and dpkg -l | grep -v 'ii' shouldn't show anything left.
Any obsolete packages that you cannot reinstall again should be removed.
Make sure the system boots cleanly on the Debian Linux i386 install.
Make a backup. That you are confident you can restore from and get a working system again.
Record the package status of the i386 system, for cross reference later on (eg, confirming the same packages got reinstalled, and the same package are marked as "automatic dependencies"):
apt-mark showauto >i386-auto-packages
dpkg --get-selections >i386-package-selections
Switch to dual architectures, and switch to a 64-bit Linux Kernel
Enable the amd64 architecture as a foreign architecture:
dpkg --print-architecture
sudo dpkg --add-architecture amd64
dpkg --print-foreign-architectures
sudo apt-get update
The base architecture at this point should be i386, but you should
be able to install amd64 architecture packages by explicitly specifying
them at this point.
Then install the 64-bit amd64 kernel, and boot into that:
sudo apt-get install linux-image-amd64:amd64
sudo update-grub
sudo shutdown -r now
For this reboot (and only this reboot) you will have to interrupt the
grub automatic boot, and pick a different kernel from the menu.
Specifically you need to go into the Advanced options for Debian GNU/Linux
menu option (second menu) and pick the amd64 kernel entry which
is named something like Debian GNU/Linux, with Linux 5.10.0-26-amd64
and will be the third entry in that submenu (because the existing i386
kernel sorts before the amd64 kernel in the generated list).
Make sure the system booted into a 64-bit kernel:
uname -m
should return x86_64 if you are running a 64-bit kernel. If not,
try rebooting again and choosing the correct kernel in the grub
boot menu.
Once you have the 64-bit kernel booting, remove the equivalent i386 (32-bit) kernel, and reboot again to make sure the 64-bit kernel is booted automatically from now on:
sudo apt-get purge linux-image-686-pae linux-image-5.10.0-26-686-pae
sudo update-grub
sudo shutdown -r now
If you wish you can pause at this point indefinitely -- the Linux i386
user land should run on the Linux amd64 kernel without any substantial
problems, as this was used for a long period in the early x86_64
kernel development.
Change the primary architecture to amd64
This is one of the more complicated steps. At this point we are changing
dpkg / apt to the amd64, which changes the primary architecture from
i386 to amd64.
On Debian Bullseye, perl-base:i386 conflicts (indirectly, I think)
with perl-base:amd64. So we have to force installing perl-base:amd64
at this point. The conflict is sufficiently bad that we cannot
even use apt to download packages once dpkg / apt have been
changed to amd64. So we need to pre-download several amd64
packages first, then install them "all at once". And then let apt
fix up the conflicts any way it can, after we have installed a few
more perl packages. (We cannot install those additional packages
at the same time as perl-base:amd64 as the conflicts are too complicated
to resolve, so apt gives up.)
This combination of steps worked for me on my Debian Bullseye systems
(although apt-get -f install removed some packages important to me
on each system, which had to be put back later on):
sudo -s
apt-get clean
apt-get --download-only install dpkg:amd64 tar:amd64 apt:amd64 apt-utils:amd64
(cd /var/cache/apt/archives/ && apt download perl-base:amd64)
dpkg --install /var/cache/apt/archives/*_amd64.deb
dpkg --install /var/cache/apt/archives/*_amd64.deb
dpkg --print-architecture # Should show: amd64
dpkg --print-foreign-architectures # Should show: i386
Note the two runs of dpkg --install ... for the same packages; this
is required because some packages will not install on the first run
due to missing amd64 dependencies; but they can install the second
time around. (There might be a topological sort of those dependencies
to install them in "the right order". But for these critical packages,
there are lots of circular dependencies, so simply running the install
of these key packages twice is the most pragmatic approach.)
Once perl-base:amd64 is installed, a bunch of other Perl dependencies
used for Debian package management will be broken, so some additional
perl packages need to be upgraded at this step, including
libtext-csv-xs-perl and libencode-perl:
(cd /var/cache/apt/archives/ && apt download libperl5.32:amd64 libgdbm-compat4:amd64 libgdbm6:amd64)
(cd /var/cache/apt/archives/ && apt download perl:amd64)
(cd /var/cache/apt/archives/ && apt download libencode-perl:amd64)
(cd /var/cache/apt/archives/ && apt download libtext-csv-xs-perl:amd64)
(cd /var/cache/apt/archives/ && apt download libdebconfclient0:amd64)
dpkg --install /var/cache/apt/archives/*_amd64.deb
At this point it should be possible to get apt back to a point where
it is "happy" automatically, but do keep track of which packages it is
removing as some of them will be important to you and need to be
reinstalled later on. Those packages need to be removed at this point
to break dependency loops:
apt-get -f install
dpkg --configure -a
Switch the shell to amd64
Install the amd64 bash and dash, and run them instead of the
32-bit versions. This will pull in several other key dependencies,
and also force-remove the i386 versions, so you will have to agree
that you know this can break your system and you want to do it
anyway (Yes, do as I say!):
apt-get install dash bash
exec /bin/bash
Install the remaining amd64 replacements for installed i386 packages
Download the packages which are needed to install amd64 replacements:
apt-get --download-only -y --no-install-recommends install \
`dpkg -l | grep '^.i' | awk '{print $2}' | grep :i386 | uniq |
grep -v 'linux-image.*686-pae' |
sed -e 's/\(.*\):i386/\1:i386- \1:amd64/'`
Then install as many of the amd64 libraries as possible (since these
can mostly be installed in parallel with the i386 packages, and fix
dependency issues):
dpkg --install /var/cache/apt/archives/lib*.deb /var/cache/apt/archives/perl*.deb
apt-get -f install
dpkg --configure -a
(we also install the outstanding perl packages at this point for certainty).
Then we can install the majority of the remaining amd64 packages:
dpkg --get-selections | grep :amd64 | cut -f 1 -d : | tee /tmp/64-bit
dpkg --get-selections | grep :i386 | grep -vf /tmp/64-bit | cut -f 1 -d : |
xargs -I {} sh -c 'ls /var/cache/apt/archives/{}*_amd64.deb' | uniq |
tee /tmp/packages-to-install
dpkg --install `cat /tmp/packages-to-install`
which might take a second pass to be able to install all the packages due to dependency issues:
dpkg --get-selections | grep :amd64 | cut -f 1 -d : | tee /tmp/64-bit
dpkg --get-selections | grep :i386 | grep -vf /tmp/64-bit | cut -f 1 -d : |
grep -v 'linux-image.*686-pae' |
xargs -I {} sh -c 'ls /var/cache/apt/archives/{}*_amd64.deb' | uniq |
tee /tmp/packages-to-install
test -s /tmp/packages-to-install && dpkg --install `cat /tmp/packages-to-install`
Tidy up the amd64 system
Check what else is left to be converted from i386 to amd64. Some
packages will still have i386 versions installed, but will also have
an amd64 version -- packages like gcc-9-base and gcc-10-base are
effectively library packages and permit parallel installs of both i386
and amd64 versions.
dpkg --get-selections | grep :i386 | grep -v lib | grep -v deinstall | wc -l
dpkg --get-selections | grep :i386 | grep -v lib | grep -v deinstall
dpkg --get-selections | grep :i386 | grep -v lib | awk '$2 ~ /^install$/ { print $1; }' | grep -v 'linux-image.*686-pae' | sed 's/:i386/:amd64/;' | grep -vf /tmp/64-bit
There may still be conflicts at this point, in which case you will need
to figure out some way to resolve them for your combination of packages;
it might involve removing a package important to you, cross grading
other packages, and then reinstalling the packages you wanted on the system
once the dependencies have been resolved properly. Inevitably if apt
cannot figure out a solution, it will be a complicated problem, so assess
carefully what you can live without temporarily to be able to make forward
progress.
Once that is done, run a normal upgrade of the amd64 system to make
sure apt is happy, and reboot the system to make sure all the running
programs are the amd64 versions:
apt-get update
apt-get upgrade
apt-get dist-upgrade
sudo update-grub
sudo shutdown -r now
Assuming the system came back up as a 64-bit system:
uname -m # Should show: x86_64
getconf LONG_BIT # Should show: 64
then clean up most of the i386 libraries next:
sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get --purge autoremove
Then I found if gnuplot was installed, I had to temporarily remove it
to make progress on the remaining cleanup (and install it again later):
HAVE_GNUPLOT=$(dpkg -l gnuplot >/dev/null 2>&1 && echo "true")
if [ -n "${HAVE_GNUPLOT}" ]; then sudo apt-get remove gnuplot; fi
before I could remove the essential i386 libraries, which hopefully
by this point your system does not need (just the amd64 versions of
those libraries). But apt / dpkg does not know this so you will
have to confirm you really want to remove these essential libraries
(Yes, do as I say!):
dpkg --get-selections | grep :i386 | grep -v deinstall | wc -l
sudo apt-get purge $(dpkg --get-selections | grep :i386 | grep -v deinstall | awk '{print $1;}')
Then you should be able to reinstall any packages that were removed to break dependency loops:
dpkg --get-selections | awk '/:i386.*deinstall/ { print $1; }' | sed 's/:i386/:amd64/'
sudo apt-get install --no-install-recommends $(dpkg --get-selections | awk '/:i386.*deinstall/ { print $1; }' | sed 's/:i386/:amd64/')
sudo apt-get -f install
sudo dpkg --configure -a
dpkg --get-selections | grep deinstall
sudo apt-get install $(dpkg --get-selections | awk '/deinstall/ { print $1; }')
Including reinstalling gnuplot again if you had to remove it above:
if [ -n "${HAVE_GNUPLOT}" ]; then sudo apt-get install gnuplot; fi
Then confirm there is nothing else that needs to be reinstalled:
dpkg --get-selections | grep deinstall
dpkg --get-selections | awk '/:i386.*deinstall/ { print $1; }' | sed 's/:i386/:amd64/'
and no i386 packages still installed:
dpkg --get-selections | grep i386
dpkg -l | grep "^i.*:386"
Cleaning up from the crossgrade
Do a regular package update to make sure there are no complaints, and then reboot to make sure you have a working system:
sudo apt-get update
sudo apt-get dist-upgrade
sudo dpkg --configure -a
sudo update-grub
sudo shutdown -r now
Assuming the system booted okay, purge the i386 packages that were
removed, and remove the i386 architecture:
sudo apt-get --purge autoremove
sudo dpkg --remove-architecture i386
dpkg --print-architecture # Should show: amd64
dpkg --print-foreign-architectures # Should return nothing
Then make sure that dpkg / apt are happy:
dpkg --audit
dpkg -l | grep -v "^ii"
and if the package state is tidy reboot again as a plain amd64 system:
sudo update-grub
sudo shutdown -r now
Ensure your production system is production ready again
Check for any other packages that might have been removed as dependencies
and need to be installed, by comparing the i386 package list to the
amd64 package list:
dpkg --get-selections >/tmp/amd64-package-selections
diff -bw <(sed 's/:i386/:amd64/;' i386-package-selections | sort) <(sort </tmp/amd64-package-selections) | grep "<" | grep -v "686-pae" | awk '{print $2;}' | grep -v "^lib" | grep -v pkgmonkey-client | tee /tmp/missing-packages
sudo apt-get install $(cat /tmp/missing-packages)
dpkg --get-selections >/tmp/amd64-package-selections
diff -wb /tmp/amd64-package-selections <(sed 's/:i386/:amd64/;' i386-package-selections) | grep -v "linux-image" | grep "^>"
sudo apt-get install $(diff -wb /tmp/amd64-package-selections <(sed 's/:i386/:amd64/;' i386-package-selections) | grep -v "linux-image" | grep -v "pkgmonkey-client" | awk '/^>/ {print $2;}')
dpkg --get-selections >/tmp/amd64-package-selections
diff -wb /tmp/amd64-package-selections <(sed 's/:i386/:amd64/;' i386-package-selections) | grep -v "linux-image" | grep -v "pkgmonkey-client" | grep "^>"
Then check for any extra packages that got dragged in as dependencies during the upgrade which you did not have installed before and do not want, and consider if you want to remove them:
dpkg --get-selections >/tmp/amd64-package-selections
diff -wb /tmp/amd64-package-selections <(sed 's/:i386/:amd64/;' i386-package-selections) | grep -v "linux-image" | grep "^<" | grep -v binutils | grep -v 'lib.san' | grep -v 'libunwind8'
Beware that some of these additional packages will be new amd64
dependencies. Eg, strace on amd64 requires libunwind8, and
gcc on amd64 indirectly requires liblsan0 and libtsan0. It
will take some systems administration experience, and some investigation,
to determine what is still needed for production and what was temporarily
installed to meet dependencies during the cross grade.
Once you are happy that the production packages are installed, and no unnecessary packages are installed, restore the "automatically installed" marks so that future upgrades will work better:
sed 's/:i386/:amd64/' i386-auto-packages | grep -v 686-pae | xargs sudo apt-mark auto
(which uses the list of i386 automatically installed packages, created at the beginning).
Then remove any packages which are now no longer required:
sudo apt-get -f install
sudo apt-get --purge autoremove
and confirm you do not have any 386 or 686 packages left:
dpkg -l | grep '[36]86'
Architecture upgrades for specific tools
Some packages have their own architecture dependent files on disk. Particularly databases, but also other programs. You will need to figure out, before committing to the production upgrade, how to handle these situations.
On my mail servers, I found that (a) postfix got removed during the
crossgrade (replaced by exim) and had to be reinstalled, and (b) the
spamassassin tool had some C libraries that it compiles to do regular
expressions faster than in perl, which have to to be rebuilt, with
sa-compile.
In the case of spamassassin I did this with:
test -x /usr/bin/sa-compile && (
sudo sa-compile --list >/tmp/before
sudo sa-compile
sudo sa-compile --list >/tmp/after
diff /tmp/before /tmp/after
sudo service spamassassin restart
sudo service spamass-milter restart
sudo service postfix restart
)
Then once you have a final configuration, reboot the system one more time to make sure you are running your final package configuration:
sudo update-grub
sudo shutdown -r now
And watch your logs and monitoring very carefully for the next few hours to keep an eye out for other problems that need manual fixes.
From this point on, if you successfully got to a working system and
fixed any data architecture dependencies, then it should act like
any other Debian Linux amd64 install, including for any future
upgrades.