VM Hosts: Single Large Host or Few Smaller Hosts?

Question

First, I apologize if this is the wrong forum to ask this question. I was wondering if this is a ServerStack or DevOps question. In either case here it goes:

I am building a small software shop and I am trying to figure out the infrastructure and machines required for it.

I bought MSDN Pro Subscription ($600/year) as well as JetBrains All Product Subscription ($600) for the development tools, desktop and server OS licenses, etc etc...

At the moment I have a budget of roughly $1500 (Walmart Credit Card) and I am entertaining the idea of buying 3x refurbished machines (each with Core i7-3770 cpu and 16GB of RAM) at $452 a pop or buying a large server at $1120 and maxing out it's RAM capacity for another $300-$400.

The background info on this is that I want to develop a product and have a full blown development environment (Source Control, Build/Deploy/Test and Release Management, Dev, Beta, and Prod environments). Cloud services (the cheapest) will run me about $200-$300 a month, and over period of 6 months it will become a recurring liability rather than an asset (which I pay out of pocket btw).

If I buy the hardware - it will have an initial cost of about $1500 but it will last me for the duration of the development process which is at least a year or so. Beyond that, if I outgrow it's capacity, I can move to docker containers and maybe a cloud solution for the Dev, Beta, and Prod environments.

So the question is - which is better - multiple dedicated cpu's or a single large server?

1x Large VM Host (64gb RAM) to have the following VMS (from here) (4GB used by Hyper-V 2016 Server):

• 1x - 8GB RAM - TFS - (Git, Active Directory, SQL2016, Sharepoint, IIS)
• 1x - 4GB RAM - TFS - Build Controller
• 1x - 4GB RAM - TFS - Release Agent, Test Agent
• 1x - 4GB RAM - DEV - Linux Mint (Postgre SQL)
• 1x - 4GB RAM - DEV - Linux Mint (Web Server/Apache)
• 1x - 4GB RAM - BETA - Linux Mint (Postgre SQL)
• 1x - 4GB RAM - BETA - Linux Mint (Web Server/Apache)
• 1x - 4GB RAM - PROD - Linux Mint (Postgre SQL)
• 1x - 4GB RAM - PROD - Linux Mint (Web Server/Apache)

Or having multiple smaller hosts:(from here)

1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):

• 1x - 6GB RAM - TFS - (Git, Active Directory, SQL2016, Sharepoint, IIS)
• 1x - 3GB RAM - TFS - Build Controller
• 1x - 3GB RAM - TFS - Release Agent, Test Agent

1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):

• 1x - 4GB RAM - DEV - Linux Mint (Postgre SQL)
• 1x - 4GB RAM - BETA - Linux Mint (Postgre SQL)
• 1x - 4GB RAM - PROD - Linux Mint (Postgre SQL)

1x Small Host - 16GB RAM (4GB used by Hyper-V 2016 Server):

• 1x - 4GB RAM - DEV - Linux Mint (Web Server/Apache)
• 1x - 4GB RAM - BETA - Linux Mint (Web Server/Apache)
• 1x - 4GB RAM - PROD - Linux Mint (Web Server/Apache)

Answer 1

A good rule of thumb is to try and avoid using one bit of logical kit (be it a single VM or a single physical host) for more than one thing.

If a physical server is going to be a VM host then that's all it should be. If a VM or a physical host is going to be a source code repository then that's all it should be etc. You can squeeze a bit extra out of your infrastructure if you consider everything to be a VM host and you then partition up those VMs to be single purpose hosts themselves.

The rationale behind this is that

1. A single host going down shouldn't impact your entire stack. Trust me this will hurt you badly as soon as you need downtime for an upgrade.
2. The cost to scale any individual component increases drastically if you want to try and scale on multiple similar servers (which will save you management dollars by reducing complexity) or it will cost you to scale out adhoc hardware as needed because you then need to deal with many different-sized hosts that are a pain to manage and require anyone managing them to know the platform intimately.

recommendation: split your infrastructure into as small a unit as is logical. If you can only afford to buy a single large bit of kit then make sure you split it into logically separate VMs or containers so that if you need to pop a bit of your infrastructure onto another server, or you need to scale stuff at different rates you can easily expand out but investing in a smaller upgrade.

Obviously the realities of software licencing, especially for windows OS's might make this cost prohibitive but, while I haven't caught up on windows licensing models in a while, I'm sure there are development centric licensing models that can suit your specific needs.

Answer 2

As I see it, the basic problem you have is that you aren't looking at the Total Cost of Ownership (TCO). While you may eventually save on the hardware costs, this is costing you in man-hours - either yours or your employees - a cost which you also pay out of pocket until you begin generating revenue.

The reason for this is this: Instead of using a ready to go system, you have to set up, install and configure the following:

• Hardware and RAID
• Version and source control
• DNS
• Firewall
• Operating system
• Database(es)
• Web server
• Build controller / test agent
• Backups

Then you have to deal with the operational maintenance of that as well. While some operational maintenance is required with in-house equipment, There is less required if you host. Consider:

Instead of sharing enterprise-grade infrastructure, you are buying white box home desktops from Walmart. This means you are accepting all of the liability and reliability issues that goes with that - drive failures, motherboards that fry, CPUs that overheat and toast, PSUs that give out and so forth - making your backups (which you don't even have to deal with if you host) that much more important. Odds are at those prices, these machines are built off of "seconds" - refurbished parts that have already failed once. There is a reason my organization purchases proper servers from Dell or HP - it's reliability.

This early in your software shop's life it is important to have focus on a single strategic vision - to produce great software (whatever software that is). In order to do that most effectively, you need to minimize distractions. You don't want to be in the hosting business too. It will distract you from making great software and that setup and management will cost you a great deal more than it would cost to simply have it hosted.

Figure out an estimate of how long it would take for you to set up and build what your hosting provider already has using the components you are looking at. Then multiply times your (or your employee's) hourly rate and factor that into your cost. At 300 a month, you are nearly halfway through your year. If your figured set up costs are greater than 1500 to 2000 dollars, then it is actually cheaper to host - plus less headache and you got a true year of development time instead of losing 2 weeks to a month (or more!) to setting up your environment.

After that year, you can still move in-house on the hardware - you were planning on possibly needing to migrate anyway.

---

But finally, to answer your question, use the three smaller hosts. You are clearly virtualizing anyway and this provides you redundancy. If your single large hosts crashes or reboots, it affects your entire system. If you have a problem with one instance, it affects 1/3rd of your environment and you might even be able to seamlessly migrate your VM to another host if your system is properly configured.