Homelab Server Hardware

January 27, 2019January 27, 2019 Timothy StewartNo Comments

In my last post I I talked about a refurbished rackmount server I picked up to replace my old gaming desktop which was being used as a server for the last few years. I outgrew my last server, both in hardware and workload, and now it was time to prepare for the next few years starting with hardware.

CPU & RAM

There isn’t much more here to say than what was said in my last post, nor much more I can add to this server. The only possible upgrade would be CPU however it wouldn’t make much sense to pay to upgrade the processors in this server. Here are the specs.

Processors: 2x 2.93 GHz X5670 12-Cores Total
Memory: 18×8 GB (144GB) RAM

Storage

Data Volume

The new homelab server came shipped with 6 x 2 TB drives, which is a total of 12 TB of storage. In total, this is more than enough space than I need however I need some redundancy. It also came with a PERC H700 RAID adapter, so all I needed to do now was choose the right configuration for my array. My options were:

RAID 0 (Stripe set)
- Usable capacity: 12 TB
- Speed gain: 6x read speed gain, 6x write speed gain
- Tolerance: None
RAID 5 (Stripe set with parity)
- Usable capacity: 10 TB
- Speed gain: 5x read speed gain, no write speed gain
- Tolerance: 1 drive failure
RAID 0+1 (Mirror of stripes or “RAID 10”)
- Usable capacity: 6 TB
- Speed gain: 6x read speed gain, 3x write speed gain
- Tolerance: 1 drive failure
RAID 6 (Stripe with double parity)
- Usable capacity: 8 TB
- Speed gain: 4x read speed gain, no write speed gain
- Tolerance: 2 drive failure

Like most things, there are trade-offs with each option. I had to balance space, with speed, with fault tolerance. RAID 0 isn’t really an option for me, no tolerance means if one drive dies I lose the entire array. Most of the data stored on this volume is critical data. It’s pictures, documents, music, and all other digital junk I have been collecting over the years. It’s also a location for all the TV I record over-the-air with Plex. So, in short, I need the most space, with some decent read/write speeds, and some fault tolerance. I ended up choose RAID 10. Looking back, RAID 6 would have also been a good option however I (thought) I needed the write performance when recording multiple shows with Plex. It’s also worth mentioning that even though I have some drive redundancy, it isn’t a backup solution. I do use a backup service to back up my critical data. I will go into detail on that in a later post.

So that took care of my “slow” data volume – my long term/slow moving/large file store, but what about fast moving, performant storage? SSDs are the obvious choice however I have limited physical space and wiring within the server chassis. I narrowed down my options to 2 parts – Host OS and Guest OS. Since I will be using a Hypervisor, I will need some storage for it (albeit not the fastest) and then fast storage for all of my guest OSes.

Host OS Volume

The host OS I chose was Proxmox, more about this in a future post, however I needed drive space for this. If you recall, my server came with all storage slots taken. There was however this (near) useless optical drive. That’s when I got the idea to replace it. I remembered reading about dummy slots for laptops that would allow you to remove the DVD drive and replace it with a drive carriage that would house a hard drive. So that’s exactly what I did. I picked up this hard drive caddy so I could replace the DVD drive with a solid state. It worked out perfectly. It was as simple as removing a few screws, popping in my hard drive, then sliding it back into the server. Worth nothing that this is a SATA II controller, so I decided to put an older SSD drive here. This is really just housing the operating system, which does limited reads and writes after booting so it was a perfect fit.

Guest OS Volumes

Now that I have my storage volume and host OS volume taken care of it was time to figure out my guest OS volume. This will house Windows and Linux guest operating systems and will need fast random read/write. Again, this server doesn’t have any available drive slots or power and my options were limited. After doing some research, I realized that the server came with quite a few options for I/O:

Two x8 and two x4 PCIe Gen2 slots
or One x16 PCIe slot and two x4 PCIe Gen2

I toyed with the idea of getting drive caddies for traditional SSD drives that could pull power and I/O from PCIe slots but then decided on something a little different. I stumbled upon these adapters that would allow you to attach an NVMe M.2 drive and use it in a PCIe slot. I figured these would work perfect. This would give me access to fast NVMe drives with out the need for power or cabling, something this server lacks. I picked up 2 of the caddies along with 2 500 GB Crucial NVMe drives. This would allow me to divide up my guest OS workloads between 2 drives to limit I/O on each. Although the jobs are critical that run on these drives, the data is not. I don’t back up any of the data on these drives except for config and a few databases.

You can see the NVMe drives below in the upper left, connected to the PCIe riser. You can also see the optical drive replacement on the upper right, however you can’t tell there’s a drive inside!

Peripherals

GPU

If you look closely, you can see that I have a video card tucked in my server. This was one of the biggest challenges for this server and one of the reasons I wanted to upgrade my previous one. Gaming on a server? Not really. Mining Bitcoin? Nope. (I did however learn a ton about building a mining rig through all this!) Running an interactive stream bot that encodes video on-the-fly on Twitch? You guessed it! Without going into detail as to why I have a video card in my server, I had to overcome a fee challenges with getting a video card in here.

No power pins for a video card
Cannot draw more than 25w on boot
No PCIe 16x slot

This left me with very few options…

Regarding the power, I knew that running external power to this card via adapters or splicing server wiring was not going to be an option for me. I didn’t want to run the risk or damaging the server or burning my house down. I am sure people have been able to accomplish this without doing either of the two however this is not something I wanted to pursue. I knew I had to find a video card that didn’t need an additional power pin.

Power draw was also going to be an issue however I assumed that most cards that didn’t need power from one of the power pins weren’t going to draw more than 25w on boot. This was an assumption and I had to roll this dice on this one.

My next challenge was the PCIe slot. Dell sells a 16x riser however they are crazy expensive and only turn up every so often on eBay. I didn’t want to sink more money into this server for a part that can’t be used in the future. Also, my GPU load didn’t require the full 16x bus. I could get by on 8x, which I did on my last server – so I sought out modifying the riser card. These are cheap to replace and a 16x card can run at 8x. I ended up taking a Dremel and grinding down the end of the 8x slot. I took it slow and ground the back of the slot off so that the card could still be seated. It’s difficult to explain but if you grind perpendicular to the slot (not intuitive) you can slowly remove enough plastic without damaging the board or pins. After I removed enough plastic I was able to seat the card in the slot with the other half (8x) hanging out.

So which card did I go with? I ended up choosing an MSI Nvidia GTX1050 2G OC and it worked out perfectly!

It gets all the power it needs from the PCIe slot
It doesn’t draw more than 25w of power when it boots
After modifying the PCIe riser, it runs great at 8x

Overall I am very satisfied with the hardware build. I am able to get lots of processing power and RAM, lots of slow storage, and lots of fast storage too. Adding a GPU was icing on the cake (although a requirement for my application). This hardware build is setting me up perfectly for a Hypervisor OS that can run all of my workloads, including video encoding, all in one system.

How about you? Have you ever modded the hardware in your PC/server? Ever run into a problem where you had to apply a “creative” solution?

New Homelab Server

January 12, 2019January 14, 2019 Timothy StewartNo Comments

If you know me, you know I like technology. If you know me well, you know I have a server at home. I have been running a server at home for many years. It has ranged from old Compaq PCs that were headed for the landfill to previous gaming desktops that are more than capable of running server workloads I throw at it. Each piece of hardware was always a hand-me-down from a business, a person, or even my own machines that aren’t ready to be donated yet.

I have also bounced back-and-forth between many server OSs – both Windows and Linux desktop/server. I have even run many types of Hypervisors too – from Windows Server Hyper-V to VMware’s ESXI to Proxmox / KVM Linux. Each has its pros and cons and each were chosen to address a specific need at the time.

Over the years the types of workloads I run at home have changed dramatically. From simple web sites that are nothing more than static HTML pages, to fully virtualized UTM firewalls, home security software, Docker clusters, and even Twitch streams.

I have always wanted to have my own server rack. Not sure why. Maybe it was due to all the servers I racked in my first job out of college. There was always something awesome to me about a rackmount server. Something about how nicely they fit in a rack, how well you can organized cables, and how you have a dedicated space for everything. So when I set out to acquire my next server, I knew it would be a rackmountable one.

I ended up picking up one from Amazon. It’s a Dell R710 and it’s fully loaded The price had dropped a few hundred dollars since the last time I looked at it so I decided it was time to buy. It’s also super popular with /r/homelab on Reddit.

System: Dell PowerEdge R710 6B LFF Server
Processors: 2x 2.93GHz X5670 12-Cores Total
Memory: 18x8GB(144GB)RAM
Hard Drives: 6x 2TB 3.5″ HDD
RAID: iDRAC6
Optical Drive: DVD-ROM
Power Supplies: 2x PSU
Bezel: None
Rails: None
Operating System: None

High level – lots of RAM, decent amount of space, lots of compute power (24 logical cores) and room to grow.

I feel like this was such a good deal for such a capable system. I’ve read lots of stories on Reddit where people have gotten better deals if they piece it together however I wanted to buy a fully working system that is fully compatible with each part. It came with a 90-day warranty too.

Over the last few weeks I have migrated all my data and services to this server and so far it’s running great. I settled on an OS, hardware config, added on, and have even modified it with a Dremel (scary, I know). Over the next few weeks I am going to document some of my findings, some of my decisions, and elaborate on the workloads that run on this server.

Do you run a server at home? Have a homelab? Run a raspberry pi? Let me know some of the services and workloads you run on your 24×7 machine 🙂

Common Docker Commands

January 17, 2017February 27, 2017 Timothy StewartNo Comments

Over the last few months I have used Docker more and more. Sometime things go my way, sometimes they don’t. When they don’t, I usually have to Google the command I need to run to get back on track. I decided to compile a list of common Docker commands to solve various issues I run into when working with Docker. This list will grow over time.

Run at your own risk. I am not responsible for copying & pasting without fully understanding what the implications are.

Show:

# Show running containers
docker ps

# Show all containers
docker ps -a

# Show all images
docker images

# Show running containers

docker ps

# Show all containers

docker ps -a

# Show all images

docker images

Stop:

# Stop a single container
docker stop <container id>

# Stop all running containers
docker stop $(docker ps -a -q)

# Stop a single container

docker stop <container id>

# Stop all running containers

docker stop $(docker ps -a -q)

Delete:

# Delete a single container
docker rm <container id>

# Delete all containers
docker rm $(docker ps -a -q)

# Delete a single image
docker rmi <image id>

# Delete all images
docker rmi $(docker images -q)

# Force delete a single image
docker rmi -f <image id>

# Force delete all images (if the above doesn't work)
docker rmi -f $(docker images -q)

# Delete a single container

docker rm <container id>

# Delete all containers

docker rm $(docker ps -a -q)

# Delete a single image

docker rmi <image id>

# Delete all images

docker rmi $(docker images -q)

# Force delete a single image

docker rmi -f <image id>

# Force delete all images (if the above doesn't work)

docker rmi -f $(docker images -q)

Build:

docker build -t Dockerfile .

1	docker build -t Dockerfile .

Publish:

docker docker push registry-host:5000/<your_name>/<your_repo>

1	docker docker push registry-host:5000/<your_name>/<your_repo>

Hosting a Meteor app on Ubuntu using Node, Mongo, and PM2

March 21, 2016December 30, 2016 Timothy Stewart1 Comment

Recently I had to start hosting a Meteor app on Ubuntu. The process is pretty straightforward however there are a few gotchas. This post is really just to help me remember in the future.

First, clone your repo on the target server or move the source files there before you compile/build the app. This is important if you are using a different OS than what your server is. I ran into issue when compiling it on OSX and moving it to Ubuntu.

After moving the files, install meteor.

curl https://install.meteor.com/ | sh

1	curl https://install.meteor.com/ \| sh

Then compile your app

meteor build .

1	meteor build .

It will create a archive for you.

Extract it to the directory you will host the app

tar -zxf myapp.tar.gz

1	tar -zxf myapp.tar.gz

cd into the server folder

cd /var/www/myapp/bundle/programs/server

1	cd /var/www/myapp/bundle/programs/server

install node modules

npm install

1	npm install

install pm2

npm install -g pm2

1	npm install -g pm2

cd back to the app bundle dir

cd /var/www/myapp/bundle

1	cd /var/www/myapp/bundle

create a process.json file

nano process.json

1	nano process.json

add your configuration

{
  "apps": [
    {
      "name": "myapp",
      "script": "./bundle/main.js",
      "log_date_format": "YYYY-MM-DD",
      "exec_mode": "fork_mode",
      "env": {
        "PORT": 3000,
        "MONGO_URL": "mongodb://127.0.0.1/myapp",
        "ROOT_URL": "https://myapp.example.com/",
        "BIND_IP": "127.0.0.1"
      }
    }
  ]
}

{

"apps": [

{

"name": "myapp",

"script": "./bundle/main.js",

"log_date_format": "YYYY-MM-DD",

"exec_mode": "fork_mode",

"env": {

"PORT": 3000,

"MONGO_URL": "mongodb://127.0.0.1/myapp",

"ROOT_URL": "https://myapp.example.com/",

"BIND_IP": "127.0.0.1"

}

]

}

create a shell script

nano run.sh

1	nano run.sh

add bash script

#!/bin/sh

# starts pm2 and watches the process

pm2 start process.json -i max --watch

#!/bin/sh

# starts pm2 and watches the process

pm2 start process.json -i max --watch

make it executable

chmod +x run.sh

1	chmod +x run.sh

run the bash script

./run.sh

./run.sh

run pm2 startup

pm2 startup

1	pm2 startup

this will output a command to run, run it

save current process

pm2 save

pm2 save

All set! Check the status of your app

pm2 list

pm2 list

You should see something similar to this

Also, here is a sample nginx site configuration using Let’s Encrypt SSL

server_tokens off; # for security-by-obscurity: stop displaying nginx version

        # this section is needed to proxy web-socket connections
        map $http_upgrade $connection_upgrade {
            default upgrade;
            ''      close;
        }

server {
        listen 443 ssl;
        root /var/www/myapp/bundle/;
        #index index.html index.htm;
        server_name myapp.example.com;
        ssl_certificate /etc/letsencrypt/live/myapp.example.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/myapp.example.com/privkey.pem;

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_prefer_server_ciphers on;
        ssl_dhparam /etc/ssl/certs/dhparam.pem;
        ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SH$
        ssl_session_timeout 1d;
        ssl_session_cache shared:SSL:50m;
        ssl_stapling on;
        ssl_stapling_verify on;

        add_header Strict-Transport-Security "max-age=31536000;";


   # Forward to meteor server                                                                                                       $
    location / {
         proxy_pass http://localhost:3000;
         proxy_http_version 1.1;
         proxy_set_header Upgrade $http_upgrade;
         proxy_set_header Connection "upgrade";
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location ~ /.well-known {
         allow all;
    }
}

server_tokens off; # for security-by-obscurity: stop displaying nginx version

# this section is needed to proxy web-socket connections

map $http_upgrade $connection_upgrade {

default upgrade;

'' close;

}

server {

listen 443 ssl;

root /var/www/myapp/bundle/;

#index index.html index.htm;

server_name myapp.example.com;

ssl_certificate /etc/letsencrypt/live/myapp.example.com/fullchain.pem;

ssl_certificate_key /etc/letsencrypt/live/myapp.example.com/privkey.pem;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_prefer_server_ciphers on;

ssl_dhparam /etc/ssl/certs/dhparam.pem;

ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SH$

ssl_session_timeout 1d;

ssl_session_cache shared:SSL:50m;

ssl_stapling on;

ssl_stapling_verify on;

add_header Strict-Transport-Security "max-age=31536000;";

# Forward to meteor server $

location / {

proxy_pass http://localhost:3000;

proxy_http_version 1.1;

proxy_set_header Upgrade $http_upgrade;

proxy_set_header Connection "upgrade";

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

}

location ~ /.well-known {

allow all;

}

Connecting Physical to Virtual with my Raspberry Pi

February 6, 2016December 30, 2016 Timothy StewartNo Comments

Over the last few weeks I have tinkered with a Raspberry Pi 2 and I have to say it has been pretty fun. I built my Pi with a Zebra case, 32GB micro sd card, a few heat sinks, and a bread board kit so I can easy connect things to my GPIO. I also picked up a TP-LINK TL_WN725N USB NIC for WiFi. I chose this one specifically because it has drivers for both Linux and Windows.

I have gone back-and-forth on which operating system I am going to run on this little device. I have tested Raspbian and Windows 10 IoT core. I ended up settling on both. I have 2 microSD cards, one with Windows 10 IoT core, and one with Raspbian, RetroPie, and OSMC. I used BootBerry boot loader to install multiple version of Linux on the second microSD card.

So far I am sticking with Windows 10 IoT Core, a few reasons:

I am much more proficient at C# than I am python or any other compiled language
Windows 10 IoT can run python and even Nodejs if I choose
Using Visual Studio to deploy and debug directly to my Raspberry Pi is awesome – I am use to this workflow from developing Windows 8/10 and Windows Phone apps

This isn’t an exhaustive list nor is it meant to help anyone make any decisions on what to use, it is just my preference – for the moment at least. We’ll see how things go.

I am hoping to build an LED panel that I can push messages to over http. I have made progress over the last 2 weeks. You can check out some of my code here:

My GitHub Repos

Here are some videos of my progress

Turning on a light for the first time

Scrolling text on my small LED panel

Windows 10 Launches Tomorrow. Get It Free Today.

July 28, 2015July 28, 2015 Timothy StewartNo Comments

This my Tech PSA for everyone out there not following Windows 10 or tech news in general. Windows 10 launches tomorrow and if you are running Windows 7 or 8, you get it free. So how do you get it? Easy.

Look in the system tray for the Windows logo. For those less tech savvy, that’s the area in the lower right next to the clock.

Click on the Windows logo and follow the prompts to reserve your copy.

Windows 10 should start downloading in the background and as soon as it is released tomorrow you will be able to start the upgrade.

If you’re not sure if Windows 10 is right for you, you can check out all the new features here. My answer is: If you’re running Windows, yes, you want Windows 10.

Push Notifications using Windows Azure

April 9, 2015April 9, 2015 Timothy StewartNo Comments

I finally have push notifications working in Space Weather App from Windows Azure. Notifications for all 4 platforms work using one service bus. This simplifies my architecture and support since all of my infrastructure and services already live in Azure. No more costly 3rd party services.

Space Weather App and Ionic

March 24, 2015June 16, 2015 Timothy StewartNo Comments

We made it to the Ionic Framework showcase!

Space Weather App for Android, iOS, and Windows

March 16, 2015June 16, 2015 Timothy StewartNo Comments

Today I launched Space Weather App for Android, Apple iOS, and Windows. This means it will run on Android phones & tablets, iPhones, iPads, iPods, and Windows phones & tablets. You can check it out here http://www.spaceweatherapp.com/

Delete cookies for a single site or domain in all browsers

September 17, 2014 Timothy Stewart1 Comment

Sometimes when debugging a web site you are working on, you want to clear out cookies for a single site however you don’t want to clear out all of your cookies. Some cookies you want to leave there, like your cookies that are set by Gmail/Facebook/Twitter/Outlook.com for 2 factor authentication or many other reasons. Up until now I have cleared them all out each time I need to clear cookies for a single web site I am working on. Here’s how you can clear cookies for a single domain and keep those time saving cookies.

IE

Hit F12
Click on Network
Click Clear Cookies for domain

Chrome

Click the Chrome menu Chrome menu on the browser toolbar.
Select Settings.
Click Show advanced settings.
In the “Privacy” section, click the Content settings button.
In the “Cookies” section, you can change the following cookies settings:
To delete a specific cookie, hover over the site that issued the cookie with your mouse, then click the X that appears on the right corner.

Firefox

Click the menu button and choose Options
Select the Privacy panel.
Set Firefox will: to Use custom settings for history.
Click Show Cookies…. The Cookies window will appear.
In the Search: field, type the name of the site whose cookies you want to remove. The cookies that match your search will be displayed.
Select the cookie(s) in the list to remove and click Remove Cookie.

Safari

Choose Safari > Preferences, and then click Privacy.
Click Details.
Select one website that stores cookies, and then click Remove
When you finish removing websites, click Done.

Techtronic

my experience with technology