question - I'm working on a project that involves heavy computation and will continually grow in terms of the database it works with. I want to build a small cluster (NUCs I'm thinking) the run a private cloud to run this off of. The code really will only need the CPUs for calculations, and the database will need the disk space so this architecture seems like a good way to get the most out of the hardware and easily add additional nodes as needed (as the database grows and/or more models require more computation).
I'm confident on my coding end for this and I've basically already built the database structure in Postgres - what I'm struggling with is whether or not I can scale this database out horizontally or not. I don't need full replication (though partial would be nice so one failed node doesn't result in data not being available), I just really want to be able to scale out the size of this thing and easily add hardware and/or more VMs as needed. I've seen stuff like pg-pool and pg_shard - not sure if that's what I need - just want to be able to take my existing database structure and spin up additional servers with additional disk space to house the additional data as needed. Any ideas?
