The Home Lab

When I first started this project, I wasn’t really expecting it to turn into what it did. Here I thought this was not a common thing to do, but was I wrong. I have developed some good friendships along the way while working on my team, and in that, there have been several resources shared between us all. A good buddy of mine showed me a reddit page that has a bunch of projects people have done from home. A lot are very inspirational. There are some pretty clever and smart people out there in the cyber metaverse. I have made pretty solid progress, but I look to continue to improve and make it better where I can.

I will cover what my setup is like which may help inspire you to build something of your own. Ebay, Mecari will be your friend. It is a lot more cost effective than you think, but I would advise you to stay within your means and not go for gold too fast.

A good source of information and to see some really cool lab setups can be found on reddit: https://www.reddit.com/r/homelab/

The Lab Infrastructure

I have accumulated a lot of equipment over the course of 3 years. This has been a long project, and it should be unless you have boat loads of money. For those of us like most Americans, (Making it) you have to be fiscally responsible no matter how shiny the object.

The heart beat of the lab are the servers, the 1st one being a Dell r820. This has 192GB of RAM, and 16 200GB SSDs for about 2.4TBs of space, 4 Intel Xeon e5-4650 Processors for 64 logical processors. It also has a 1GB Quad NIC card. Is that enough? Of course it isn’t, give me all the power. I was then fortunate enough to get my hands on a Cisco UCS 240 M3 which has a lower CPU thread count, but not all that bad. This server has 2 Intel Xeon E5-2690 v2 CPUs (10 Core, 20 threads) for a total of 40 logical processors, 256GB of DDR3 1600MHz RAM and roughly 11TB of on board storage. I picked up a barebones Dell r620 and had to add RAM, CPUs, Hard drives and a raid card. I have this running 2x Intel Xeon e5-2690 v2 CPUs (10 Core, 20 threads) for a total of 40 logical processors, 256GB of DDR3 1833MHz RAM and 8x 300GB SAS drives in raid 5 for 1.6TB of space. The last and the least, but still mighty Dell T5610. This thing holds its own. I use it mainly for hosting my web services like Mattermost and Nextcloud. The Dell T5610 has 2 Intel Xeon E5 2690 v2 (10 Core, 20 threads) for a total of 40 logical processors, 128GB of DDR3 1833MHZ RAM, and a not to shabby 2.8 TB of storage space in a raid 1 Configuration. This all connects to a Cisco 3750X 48 port with PoE (Not necessary) but why not. My servers are trunked to the Cisco Switch which has a point to point connection to my ASUS 1750 Router which is a DMZ Router between my lab environment and physical fan-less pfSense Firewall.

This is what makes the magic work!!

VMWare

I am running this whole setup with a combination of a lot of resources, mostly all salvaged. I run VMWare on a number of different devices that is all managed by vCenter. I built a Distributed Switch and migrated the vmkernels and VMs to run off the Dswitch for more advanced options and port mirroring control. Can you pull it off without vCenter? Yes, but why? All of the esxi servers are running version 6.7 along with vCetner as an appliance. The vCenter appliance is running on a Gigabyte Brix with an Intel i5 processor and 16GB of RAM with a 128GB Solid State Hard Drive. That is what makes up my 3rd esxi server. My Father in-law had a Dell T5600, boy was I excited. He gave it to me to add to the two that I currently have which just of course made it all better. I would have to say, overall, I have been extremely fortunate in this lengthy process.

pfSense

I absolutely love pfSense. It is the best opensource firewall I have used hands down. The development team has really made a very user friendly firewall supported very well by the community. There are tons of things that pfsense gives you the ability to do, and that is run a lot of additional packages and relatively inexpensive hardware. It just works very well. It runs on FreeBSD 11.3 at the moment, and Intel NICs seem to have the best performance. I have a main firewall for the home and then I directly connect to a Lab DMZ Router for segmentation. I also run multiple lab firewalls, all pfSense. This firewall is awesome and it makes setting up a VPN into the network really simple. I use it for a simple Router on a stick, IPS/IDS, HA Proxy, VPN, DNS Blocker, IP Reputation Blocker and it handles all of it nicely.

Environment

I am still in the process of trying to figure out how what type of simulated network I want to run. Here is what I currently have:

  • Domain Controllers for Lab Network (Windows Server 2016)DC1 / DC2 – DNS
  • piHole DNS 1 / 2 (All traffic forwarded to piHole DNS and filtered at the main firewall) – Runs on 2x Pi 3B’s
  • Docker Server (Runs Portainer on a Fanless PC)
  • Guacamole Server (Clientless RDP,VNC,SSH server through a webrowser) – Runs on a Pi 4
  • Windows Analysts VMs
  • Linux Analysts VMs
  • Kali Pen Testing VMs
  • Customer simulated ServicesCustomer WorkstationsWindows
  • Linux
  • Customer Services NetworkActive Directory (Domain)
  • Exchange (Planned)
  • File Services (Planned)
  • Traffic Generation (Planned)
  • Intrusion Detection (Planned)

Self Hosted Cloud Service

Nextcloud is pretty awesome. They offer you the ability if you have the means to build your own personal Nextcloud server. So, after much thought and seeing how nice it was for a friend of mine, I decided, why not. I went the less traditional non-docker version and decided to go the traiditonal server build with using nginx as a reverse proxy, building out mysql database and then throwing up a sub domain on cloudflare so that I could reach it internally with the use of pfsense HA proxy. I have fully ditched my reliance on Google Drive and now I am using my own Nextcloud server. It is pretty nice because they make it easy for you to brand it and add your own metadata. I have to say, I am impressed. I installed the windows 10 application and it syncs beautifully. It integrated itself with nice contextual menu options and I couldn’t be more happy. I have 20GB used of my personal 100GB abd have access on all of my devices just like I would with Google Cloud. My data is a bit more safeguarded versus Google scraping it and just selling it.

Lab Management with Portainer and Dash Machine

Dashmachine is a docker image that you can run in Docker without the need for portainer, but portainer is pretty awesome. It gives you access to templates that you can load from github or create your own. I found a template and modified it to pull the version of Dashmachine that I wanted. Dashmachine is awesome, but takes a little bit of digital patience. I absolutely love it. It is essentially a fancy favorite places in a url, so, again….why not? The first rule of cyber and network management is to just look cool.

Docker Machine (Running Portainer Frontend)

Dash Machine Network Management Dash

Guacamole (Apache Guacamole)

This is just the icing on the cake. This has enabled me to reach my lab environment from any location, no matter where I am in the world. Keep in mind, you want to be accessing this from a strong network connection with low latency and high speed. This will severely decrease performance. With that being said, I have a registered domain name for this. I use pfsense as a HA proxy to enable encrypted SSL connections from destinations that I allow. This allows me to access the full power of my lab with a web browser. You may be asking why? Why don’t you just use a VPN? Cause, why not? What if I do not have my personal laptop or better yet, what if I do not have access to my VPN? Then what? That is right, Guac it up.

Below is an example of what it looks like:

Network Security Monitoring

This does not require a whole lot to get setup and going. The way I went about it is getting a pretty inexpensive switch such as a TP-Link 16 Port port (TL-SG1016DE) or a HP 1810-24G that has the capability to mirror ports. I am mirroring the connection from my firewall (The gateway) to my network sensor which is a Virtual Machine running my Dell T5600. I have been testing out Hybrid Hunter from Security Onion. They had done a lot of work to get it to where it is, and I must say, I really love some of the Dashboards built for Kibana. Here are the services that run on mine and the Hardware assigned to the Virtual Machine

Hardware aka VM Specs

  • CPUs – 16
  • RAM – 32 GB
  • HD – 4 TB
  • 1 Virtual NIC
  • 1 Virtual promiscuous Port (Connection to Mirrored port)

Security Onion 2

Services that Run on my Security Onion 2 Instance.

  • Full Packet Capture 
  • Protocol Analysis and Metadata via Zeek.
  • Signature Based Alerting via Suricata.
  • Data Storage, Indexing, and Search via Elasticsearch.
  • Data UI and Visualization via Kibana.
  • Incident Response Platform via The Hive
  • Host Security Monitoring via Wazuh and OS Query/fleet
  • The Cyber Swiss Army Knife via CyberChef
  • Security – The system is developed and tested to run with SELinux enabled.

Network Dash in Kibana

SSH Dashboard in Kibana

Grafana System Monitoring