Reverting Debian from Sid to Stable

I've been running a personal server for quite a while in our house since I've cut cable. There were a couple things that I absolutely needed to get right before I would get a high enough WAF (wife acceptance factor) to make it usable as a daily streaming platform for TV. Namely...

• It needs to be stable. No if's, and's, or but's about it. She shouldn't have to deal with any server instability
• It needs to have TV shows in a timely manner after airing
• It needs to be something accessible from any device on our home network
• It needs to be fast. Buffering shouldn't exist

While I've done quite a good job with our current setup for items two, three, and four, item one has always come back to bite me. We're currently on our third iteration of this server.

Background

The first go-around was Arch Linux with a software RAID5 (13TB of usable storage). That setup worked great until a bad combination of packages left me with a kernel panic-ridden server for a few days. That spurred a reinstall and (because of my inexperience with software RAID) a loss of all data.

The second go-around was a FreeBSD install with ZFS RAIDZ array for storage (still 13TB of storage). This one rolled over one day due to what I suspect was yet another bad update that I couldn't resolve - no matter what I did, I couldn't get it to boot.

The last (and current) iteration consisted of Debian Sid with a BTRFS storage array in RAID5. Probably not the best decision (especially given my track record so far), but it hasn't been too bad yet. Well, until last weekend.

I decided to update the kernel to linux-image-4.11.0-1-amd64 (which, so far, had no issues at all). All was smooth until we started streaming shows through Plex, which after a few minutes of streaming (and transcoding the media to our Chromecast in our living room), caused a kernel panic every time. I attempted to roll back to the previous package, linux-image-4.9.0-3-amd64, but discovered it was no longer present in the sid repositories. It was probably overdue, but it was time to revert back to the recently released stable in Debian Stretch.

Reverting

Around fifteen minutes of Googling turned up very little in the way of moving Debian from unstable to stable. Undeterred, I took the steps from How to get back from “testing” to “stable” - Kernel downgrade and started my downgrade.

After some work in /etc/apt/sources.list adding new entries for Stretch and commenting out entries for Sid, I ended up with something like this:

## sid
#deb http://ftp.us.debian.org/debian/ sid main
#deb-src http://ftp.us.debian.org/debian/ sid main

#deb http://ftp.us.debian.org/debian/ sid contrib non-free



## stretch
deb http://deb.debian.org/debian stretch main contrib non-free
deb-src http://deb.debian.org/debian stretch main contrib non-free

deb http://deb.debian.org/debian stretch-updates main contrib non-free
deb-src http://deb.debian.org/debian stretch-updates main contrib non-free

deb http://security.debian.org/ stretch/updates main contrib non-free
deb-src http://security.debian.org stretch/updates main contrib non-free

deb http://ftp.debian.org/debian stretch-backports main contrib non-free

I did have to make some updates to /etc/apt/preferences, although I should really just be using an individual file in /etc/apt/preferences.d for the Stretch package pinning. I don't know that after this downgrade I need to keep the pin in preferences or not, but for now, my /etc/apt/preferences looks like:

Package: *
Pin: release a=stable
Pin-Priority: 1001

Moment of truth

After completing those updates, the final commands were pretty simple...a clean update from sudo apt-get update, downgrading most of the packages with sudo apt-get upgrade, and a final downgrade with sudo apt-get dist-upgrade. While all of this worked perfectly (and it rebooted smoothly after this), it still didn't resolve my kernel version issue. I was able to just use apt to reinstall the correct kernel version (sudo apt-get install linux-image-4.9.0-3-amd for those interested) and had no issues there.

The tricky part, however, was removing the existing kernel. Without rebooting into the previous kernel, attempting to remove the package (aptitude remove) backfired pretty hard and threw a really nice warning about removing the currently running kernel. This was a quick fix though; just reboot using the 4.9.0 kernel and then remove the 4.11.0 kernel package.

And that's it. Not a whole lot of magic; it probably helped that Stretch was still pretty close to Sid in terms of package versions. I probably wouldn't have tried it a few weeks or a month further out from the Stretch release. In that case, I probably would've cut to testing/buster and just settled for the slightly-ahead-of-stable changes living in testing.

Results

I've still got some streaming issues with Plex, namely transcoders crashing when attempting to stream out to a Chromecast. But, even then, it isn't causing a kernel panic, which is a definite step in the right direction. There are some more transcoding settings I can play with, my wife is using other streaming options in the interim, and we're in a much better place for stability.