cross-posted from: https://lemmy.dbzer0.com/post/5911320

The complete guide to building your personal self hosted server for streaming and ad-blocking.

Captain’s note: This OC was originally posted in reddit but it’s quality makes me wants to ensure a copy survices in lemmy as well.


We will setup the following applications in this guide:

  • Docker
  • AdguardHome - Adblocker for all your devices
  • Jellyfin/Plex - For watching the content you download
  • Qbittorrent - Torrent downloader
  • Jackett - Torrent indexers provider
  • Flaresolverr - For auto solving captcha in some of the indexers
  • Sonarr - *arr service for automatically downloading TV shows
  • Radarr - *arr service for movies
  • Readarr - *arr service for (audio)books
  • lidarr - *arr service for music
  • Bazarr - Automatically downloads subtitles for Sonarr and Radarr
  • Ombi/Overseer - For requesting movies and tv shows through Sonarr and Radarr
  • Heimdall - Dashboard for all the services so you don’t need to remember all the ports

Once you are done, your dashboard will look something like this.

Heimdall Dashboard

I started building my setup after reading this guide https://www.reddit.com/r/Piracy/comments/ma1hlm/the_complete_guide_to_building_your_own_personal/.

Hardware

You don’t need powerful hardware to set this up. I use a decade old computer, with the following hardware. Raspberry pi works fine.

Hardware

Operating system

I will be using Ubuntu server in this guide. You can select whatever linux distro you prefer.

Download ubuntu server from https://ubuntu.com/download/server. Create a bootable USB drive using rufus or any other software(I prefer ventoy). Plug the usb on your computer, and select the usb drive from the boot menu and install ubuntu server. Follow the steps to install and configure ubuntu, and make sure to check “Install OpenSSH server”. Don’t install docker during the setup as the snap version is installed.

Once installation finishes you can now reboot and connect to your machine remotely using ssh.

ssh username@server-ip 
# username you selected during installation
# Type ip a to find out the ip address of your server. Will be present against device like **enp4s0** prefixed with 192.168.

Create the directories for audiobooks, books, movies, music and tv.

I keep all my media at ~/server/media. If you will be using multiple drives you can look up how to mount drives.

We will be using hardlinks so once the torrents are downloaded they are linked to media directory as well as torrents directory without using double storage space. Read up the trash-guides to have a better understanding.

mkdir ~/server
mkdir ~/server/media # Media directory
mkdir ~/server/torrents # Torrents

# Creating the directories for torrents
cd ~/server/torrents
mkdir audiobooks  books  incomplete  movies  music  tv 

cd ~/server/media
mkdir audiobooks  books  movies  music  tv

Installing docker and docker-compose

Docker https://docs.docker.com/engine/install/ubuntu/

# install packages to allow apt to use a repository over HTTPS
sudo apt-get update
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
# Add Docker’s official GPG key:
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
# Setup the repository
echo \
  "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Install Docker Engine
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
# Add user to the docker group to run docker commands without requiring root
sudo usermod -aG docker $(whoami) 

Sign out by typing exit in the console and then ssh back in

Docker compose https://docs.docker.com/compose/install/

# Download the current stable release of Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
# Apply executable permissions to the binary
sudo chmod +x /usr/local/bin/docker-compose

Creating the compose file for Adguard home

First setup Adguard home in a new compose file.

Docker compose uses a yml file. All of the files contain version and services object.

Create a directory for keeping the compose files.

mkdir ~/server/compose
mkdir ~/server/compose/adguard-home
vi ~/server/compose/adguard-home/docker-compose.yml

Save the following content to the docker-compose.yml file. You can see here what each port does.

version: '3.3'
services:
    run:
        container_name: adguardhome
        restart: unless-stopped
        volumes:
            - '/home/${USER}/server/configs/adguardhome/workdir:/opt/adguardhome/work'
            - '/home/${USER}/server/configs/adguardhome/confdir:/opt/adguardhome/conf'
        ports:
            - '53:53/tcp'
            - '53:53/udp'
            - '67:67/udp'
            - '68:68/udp'
            - '68:68/tcp'
            - '80:80/tcp'
            - '443:443/tcp'
            - '443:443/udp'
            - '3000:3000/tcp'
        image: adguard/adguardhome

Save the file and start the container using the following command.

docker-compose up -d

Open up the Adguard home setup on YOUR_SERVER_IP:3000.

Enable the default filter list from filters→DNS blocklist. You can then add custom filters.

Filters

Creating the compose file for media-server

Jackett

Jackett is where you define all your torrent indexers. All the *arr apps use the tornzab feed provided by jackett to search torrents.

There is now an *arr app called prowlarr that is meant to be the replacement for jackett. But the flaresolverr(used for auto solving captchas) support was added very recently and doesn’t work that well as compared to jackett, so I am still sticking with jackett for meantime. You can instead use prowlarr if none of your indexers use captcha.

jackett:
    container_name: jackett
    image: linuxserver/jackett
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    volumes:
      - '/home/${USER}/server/configs/jackett:/config'
      - '/home/${USER}/server/torrents:/downloads'
    ports:
      - '9117:9117'
    restart: unless-stopped
prowlarr:
		container_name: prowlarr
    image: 'hotio/prowlarr:testing'
    ports:
      - '9696:9696'
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    volumes:
      - '/home/${USER}/server/configs/prowlarr:/config'
    restart: unless-stopped

Sonarr - TV

Sonarr is a TV show scheduling and searching download program. It will take a list of shows you enjoy, search via Jackett, and add them to the qbittorrent downloads queue.

sonarr:
    container_name: sonarr
    image: linuxserver/sonarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    ports:
      - '8989:8989'
    volumes:
      - '/home/${USER}/server/configs/sonarr:/config'
      - '/home/${USER}/server:/data'
    restart: unless-stopped

Radarr - Movies

Sonarr but for movies.

radarr:
    container_name: radarr
    image: linuxserver/radarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    ports:
      - '7878:7878'
    volumes:
      - '/home/${USER}/server/configs/radarr:/config'
      - '/home/${USER}/server:/data'
    restart: unless-stopped

Lidarr - Music

lidarr:
    container_name: lidarr
    image: ghcr.io/linuxserver/lidarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    volumes:
      - '/home/${USER}/server/configs/liadarr:/config'
      - '/home/${USER}/server:/data'
    ports:
      - '8686:8686'
    restart: unless-stopped

Readarr - Books and AudioBooks

# Notice the different port for the audiobook container
readarr:
    container_name: readarr
    image: 'hotio/readarr:nightly'
    ports:
      - '8787:8787'
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    volumes:
      - '/home/${USER}/server/configs/readarr:/config'
      - '/home/${USER}/server:/data'
    restart: unless-stopped

readarr-audio-books:
    container_name: readarr-audio-books
    image: 'hotio/readarr:nightly'
    ports:
      - '8786:8787'
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    volumes:
      - '/home/${USER}/server/configs/readarr-audio-books:/config'
      - '/home/${USER}/server:/data'
    restart: unless-stopped

Bazarr - Subtitles

bazarr:
    container_name: bazarr
    image: ghcr.io/linuxserver/bazarr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    volumes:
      - '/home/${USER}/server/configs/bazarr:/config'
      - '/home/${USER}/server:/data'
    ports:
      - '6767:6767'
    restart: unless-stopped

Jellyfin

I personally only use jellyfin because it’s completely free. I still have plex installed because overseerr which is used to request movies and tv shows require plex. But that’s the only role plex has in my setup.

I will talk about the devices section later on.

For the media volume you only need to provide access to the /data/media directory instead of /data as jellyfin doesn’t need to know about the torrents.

jellyfin:
    container_name: jellyfin
    image: ghcr.io/linuxserver/jellyfin
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    ports:
      - '8096:8096'
    devices:
      - '/dev/dri/renderD128:/dev/dri/renderD128'
      - '/dev/dri/card0:/dev/dri/card0'
    volumes:
      - '/home/${USER}/server/configs/jellyfin:/config'
      - '/home/${USER}/server/media:/data/media'
    restart: unless-stopped

plex:
    container_name: plex
    image: ghcr.io/linuxserver/plex
    ports:
      - '32400:32400'
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
      - VERSION=docker
    volumes:
      - '/home/${USER}/server/configs/plex:/config'
      - '/home/${USER}/server/media:/data/media'
    devices:
      - '/dev/dri/renderD128:/dev/dri/renderD128'
      - '/dev/dri/card0:/dev/dri/card0'
    restart: unless-stopped

Overseer/Ombi - Requesting Movies and TV shows

I use both. You can use ombi only if you don’t plan to install plex.

ombi:
    container_name: ombi
    image: ghcr.io/linuxserver/ombi
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    volumes:
      - '/home/${USER}/server/configs/ombi:/config'
    ports:
      - '3579:3579'
    restart: unless-stopped

overseerr:
    container_name: overseerr
    image: ghcr.io/linuxserver/overseerr
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    volumes:
      - '/home/${USER}/server/configs/overseerr:/config'
    ports:
      - '5055:5055'
    restart: unless-stopped

Qbittorrent - Torrent downloader

I use qflood container. Flood provides a nice UI and this image automatically manages the connection between qbittorrent and flood.

Qbittorrent only needs access to torrent directory, and not the complete data directory.

qflood:
    container_name: qflood
    image: hotio/qflood
    ports:
      - "8080:8080"
      - "3005:3000"
    environment:
      - PUID=1000
      - PGID=1000
      - UMASK=002
      - TZ=Asia/Kolkata
      - FLOOD_AUTH=false
    volumes:
      - '/home/${USER}/server/configs/qflood:/config'
      - '/home/${USER}/server/torrents:/data/torrents'
    restart: unless-stopped

Heimdall - Dashboard

There are multiple dashboard applications but I use Heimdall.

heimdall:
    container_name: heimdall
    image: ghcr.io/linuxserver/heimdall
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    volumes:
      - '/home/${USER}/server/configs/heimdall:/config'
    ports:
      - 8090:80
    restart: unless-stopped

Flaresolverr - Solves cloudflare captcha

If your indexers use captcha, you will need flaresolverr for them.

flaresolverr:
    container_name: flaresolverr
    image: 'ghcr.io/flaresolverr/flaresolverr:latest'
    ports:
      - '8191:8191'
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Asia/Kolkata
    restart: unless-stopped

Transcoding

As I mentioned in the jellyfin section there is a section in the conmpose file as “devices”. It is used for transcoding. If you don’t include that section, whenever transcoding happens it will only use CPU. In order to utilise your gpu the devices must be passed on to the container.

https://jellyfin.org/docs/general/administration/hardware-acceleration.html Read up this guide to setup hardware acceleration for your gpu.

Generally, the devices are same for intel gpu transcoding.

devices:
      - '/dev/dri/renderD128:/dev/dri/renderD128'
      - '/dev/dri/card0:/dev/dri/card0'

To monitor the gpu usage install intel-gpu-tools

sudo apt install intel-gpu-tools

Now, create a compose file for media server.

mkdir ~/server/compose/media-server
vi ~/server/compose/media-server/docker-compose.yml

And copy all the containers you want to use under services. Remember to add the version string just like adguard home compose file.

Configuring the docker stack

Start the containers using the same command we used to start the adguard home container.

docker-compose up -d

Jackett

Navigate to YOUR_SERVER_IP:9117

Add a few indexers to jackett using the “add indexer” button. You can see the indexers I use in the image below.

Indexers

Qbittorrent

Navigate to YOUR_SERVER_IP:8080

The default username is admin and password adminadmin. You can change the user and password by going to Tools → Options → WebUI

Change “Default Save Path” in WebUI section to /data/torrents/ and “Keep incomplete torrents in” to /data/torrents/incomplete/

Create categories by right clicking on sidebar under category. Type category as TV and path as tv. Path needs to be same as the folder you created to store your media. Similarly for movies type Movies as category and path as movies. This will enable to automatically move the media to its correct folder.

Sonarr

Navigate to YOUR_SERVER_IP:8989

  • Under “Download Clients” add qbittorrent. Enter the host as YOUR_SERVER_IP port as **8080,** and the username and password you used for qbittorrent. In category type TV (or whatever you selected as category name(not path) on qbittorent). Test the connection and then save.
  • Under indexers, for each indexer you added in Jackett
    • Click on add button
    • Select Torzab
    • Copy the tornzab feed for the indexer from jackett
    • Copy the api key from jackett
    • Select the categories you want
    • Test and save
  • Under general, define the root folder as /data/media/tv

Repeat this process for Radarr, Lidarr and readarr.

Use /data/media/movies as root for Radarr and so on.

The setup for ombi/overseerr is super simple. Just hit the url and follow the on screen instructions.

Bazarr

Navigate to YOUR_SERVER_IP:6767

Go to settings and then sonarr. Enter the host as YOUR_SERVER_IP port as 8989. Copy the api key from sonarr settings→general.

Similarly for radarr, enter the host as YOUR_SERVER_IP port as 7878. Copy the api key from radarr settings→general.

Jellyfin

Go to YOUR_SERVER_IP:8096

  • Add all the libraries by selecting content type and then giving a name for that library. Select the particular library location from /data/media. Repeat this for movies, tv, music, books and audiobooks.
  • Go to dashboard→playback, and enable transcoding by selecting as VAAPI and enter the device as /dev/dri/renderD128

Monitor GPU usage while playing content using

sudo intel_gpu_top

Heimdall

Navigate to YOUR_SERVER_IP:8090

Setup all the services you use so you don’t need to remember the ports like I showed in the first screenshot.

Updating docker images

With docker compose updates are very easy.

  • Navigate to the compose file directory ~/server/compose/media-server.
  • Then docker-compose pull to download the latest images.
  • And finally docker-compose up -d to use the latest images.
  • Remove old images by docker system prune -a

What’s next

  • You can setup VPN if torrents are blocked by your ISP/Country. I wanted to keep this guide simple and I don’t use VPN for my server, so I have left out the VPN part.
  • You can read about port forwarding to access your server over the internet.
  • WxFisch
    link
    fedilink
    English
    371 year ago

    I would recommend prowlarr instead of jackett for indexer management, and pihole as at least an additional blocking service but in reality it’s really all you need for use at home. I’d also strongly encourage use of a VPN on your *arr download services. I use a separate box to run Plex and then have my *arrs all running on their own VM inside if it to provide separation and allow be to more easily segregate the network traffic (as someone that doesn’t really know docker that well it “just works” for me. Also probably worth looking at how to store your media on an external target, it’s easy to quickly accumulate 10s of TBs of media and trying to store that all on the server locally is asking for trouble. Better to set everything up on a NAS to start.

    • @dan@upvote.au
      link
      fedilink
      English
      91 year ago

      pihole as at least an additional blocking service

      Why would you need PiHole? AdGuard Home is the same but with more features, for example it supports and uses DNS-over-HTTPS out of the box so your ISP can’t view and modify the DNS requests and responses.

  • @ahto@feddit.de
    link
    fedilink
    English
    301 year ago

    Since you only use Plex because of overseerr: There is a fork called Jellyseer that is compatible with Jellyfin. Haven’t tried it yet but it may be worth looking into.

  • @auf@lemmy.ml
    link
    fedilink
    English
    21
    edit-2
    1 year ago

    I found a brand new dashboarding software recently, haven’t tried myself yet but if you’re interested…

    https://github.com/gethomepage/homepage

    P.S. I found it wasn’t “brand new”. It’s been there for a while but is actively maintained and have somewhat better user interface imo

      • chandz05
        link
        fedilink
        English
        41 year ago

        I did exactly the same! I tried a few different dashboards, and Homepage was the perfect balance between customization and simplicity for me

      • RBG
        link
        fedilink
        English
        11 year ago

        How well does either of these work if you connect with a phone/mobile interface? Maybe I am looking at these the wrong way but I have been wanting to get a dashboard that I could reach via my phone that doesn’t force a desktop interface.

    • @Petter1@lemm.ee
      link
      fedilink
      English
      41 year ago

      Looks great, did you compare it to LunaSea? If yes, are there pos/cons? I can’t test because I’m on iOS right now, but would be nice to know

    • morriscox
      link
      fedilink
      English
      11 year ago

      I have Tailscale setup as the primary connection with my local SSID as secondary. Works great and the Pro version is worth it.

  • @Shepy@feddit.uk
    link
    fedilink
    English
    111 year ago

    Great guide, two things though

    I notice you mention Prowlarr, I’d probably suggest it and jackett - i ran both for a while then dropped Jackett as all my matches were coming through Prowlarr and its just less admin to add sources there once and have them add to the rest of the servarr apps

    Also, you might want to look at https://www.audiobookshelf.org/ - A great plex like app for streaming audiobooks specifically

  • @dan@upvote.au
    link
    fedilink
    English
    91 year ago

    For AdGuard Home, I’d recommend running it on two servers so that your internet doesn’t break when one of the servers is down (eg for maintenance, upgrades, etc). It runs well on a Raspberry Pi. You can use AdGuardHome-Sync to keep the config in sync between the two servers.

    • @Dave@lemmy.nz
      link
      fedilink
      English
      41 year ago

      How do you get your DNS to point to the live one?

      In my experience, the router’s primary DNS server is not used first, then the secondary. Rather your device just sends half of the requests to each. If you run two and one is down, every second request fails. Unless you can run a load balancer, but then you’re back to a single point of failure.

      • SciPiTie
        link
        fedilink
        English
        31 year ago

        The client does a fallback if one dns doesn’t answer. That’s why dns ad blockers fail if 8.8.8.8 or some other dns is added as a secondary :)

        • @Dave@lemmy.nz
          link
          fedilink
          English
          21 year ago

          Hmm I’ve definitely see clients just say no when the first fails. How do they tell the router to try the other one? Does the router send both DNS servers to the client or does the client request a lookup from the router? Maybe my router sucks.

          • SciPiTie
            link
            fedilink
            English
            21 year ago

            The router is not directly involved in a dns query except, we’ll, the routing if it’s an non local IP. The DNS ip addresses is propagated either via dhcp together with the clients or directly configured in the client. That said: most routers serve as dhcp server at the same time. Perhaps your router is configured to always provide your ISPs DNS as primary.

            How the client handles the decision which to query I honestly don’t know and I guess that’s why you and I made different experiences!

            • @Dave@lemmy.nz
              link
              fedilink
              English
              11 year ago

              My router is configured with both primary and secondary DNS pointing to my pihole, but in the past I’ve had one valid and one invalid one and the DNS lookup would fail approx half the time. It was a few years ago (but not that many). I wonder what was up with that.

      • @dan@upvote.au
        link
        fedilink
        English
        1
        edit-2
        1 year ago

        If one of them fails, the client falls back to the other one. That’s the main reason why you can configure two. I think you’re right that some OSes / devices send half the queries to each server (that’s what I’m seeing too) but the fallback logic is still there too.

  • @Decronym@lemmy.decronym.xyzB
    link
    fedilink
    English
    9
    edit-2
    1 year ago

    Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:

    Fewer Letters More Letters
    DNS Domain Name Service/System
    HTTP Hypertext Transfer Protocol, the Web
    HTTPS HTTP over SSL
    IP Internet Protocol
    NAS Network-Attached Storage
    NAT Network Address Translation
    PiHole Network-wide ad-blocker (DNS sinkhole)
    Plex Brand of media server package
    SSL Secure Sockets Layer, for transparent encryption
    VPN Virtual Private Network

    [Thread #203 for this sub, first seen 9th Oct 2023, 12:25] [FAQ] [Full list] [Contact] [Source code]

    • @dan@upvote.au
      link
      fedilink
      English
      51 year ago

      AdGuard Home supports DNS over HTTPS and uses it by default (via Quad9). This encrypts your DNS requests and ensures your ISP can’t collect the queries or modify the responses. Some ISPs build an advertising profile for you based on the sites you visit, and using encrypted DNS prevents them from doing this.

      You can use DNS over HTTPS with PiHole but it’s a lot of manual work to set it up. With AdGuard Home, you just enter the DNS server URL in the web UI, and that’s it.

    • deweydecibel
      link
      fedilink
      English
      61 year ago

      People suggest this all the time, but it’s only useful if you’re the only one who is going to be accessing your library externally. Otherwise you have to get family or friends who you let access your server to use it anytime they want to stream, and that’s far easier said than done. They have to know what they’re doing.

      It also means they can’t stream on their smart TVs or streaming boxes. You know, the things most people will want to watch this content on.

      • @dan@upvote.au
        link
        fedilink
        English
        1
        edit-2
        1 year ago

        Tailscale is a VPN. It’s built on top of Wireguard but simplifies/automates the configuration. It gives you a mesh VPN without having to configure the mesh manually - with regular Wireguard you’d have to add each peer to every other peer’s config manually.

        It provides NAT traversal too where needed (for example, if your network doesn’t have native IPv6).

        • Possibly linux
          link
          fedilink
          English
          21 year ago

          I’m aware that tailscale is a vendor locked version of wireguard. However, that doesn’t change the fact that ssh is sometimes the easiest way to get remote access.

          If you want to take the wireguard route you can either setup wireguard yourself or use something like netbird.

  • @lemming741@lemmy.world
    link
    fedilink
    English
    31 year ago

    Unsolicited plug for audiobookshelf- runs in a browser, supports multiple users, multiple libraries, and user-friendly, in active development too!

    https://www.audiobookshelf.org/docs#intro

    Separate instances of readarr for audio and text was a let-down for me. I just grab the torrent from MAM, and move it to the library folder from transmission remote. ABS picks it up when it’s finished. We don’t do THAT many books to make readarr worth it.

  • @Yerbouti@lemmy.ml
    link
    fedilink
    English
    21 year ago

    Tanks, that’s a really detailed guide. Would it also work as a home server for Nextcloud? I wanna ditch the Google office suite.

    • @ChillPill@lemmy.world
      link
      fedilink
      English
      21 year ago

      I run the snap version of nextcloud in an ubuntu VM. I know snap gets a lot of flac, but I have a day job and I don’t want to spend all my time trying to keep nextcloud running.