DevOps Stack Exchange
DevOps Stack Exchange

Questions tagged [scalability]

12 questions
7
votes
3 answers

How can continuous integration scale for very large projects/teams?

Traditionally CI systems only monitor the quality of the code in an integration branch, signalling when regressions occur. Human intervention is required for repairs. As the number of developers working on the same branch increases the risk of…
Dan Cornilescu
  • 6,780
  • 2
  • 21
  • 45
6
votes
2 answers

if the bottle neck is the number of connections, is it better to scale redis by sharding or by replication?

Background last week we had a major outage, we identified the bottle neck to be redis number of connections: Our quick fix was to scale redis from cache.m4.4xlarge to chache.m4.10xlarge (vertical scaling) However this scaling is quite…
abbood
  • 473
  • 4
  • 13
6
votes
4 answers

How can Agile methodologies, CI/CD be applied to massive, monolithic software projects with 100s/1000s of developers?

Using continuous integration represents an (almost) mandatory requirement for a software project to be agile. And without CI there can be no continuous delivery/deployment. But CI can face scalability challenges, see How can continuous integration…
Dan Cornilescu
  • 6,780
  • 2
  • 21
  • 45
4
votes
1 answer

Running multiple websites on Docker in isolated or shared database containers?

So I have been reading up on Docker, and I cannot quite get my head around this concept. There are literally dozens of tutorials on the web about spawning a Wordpress instance on Docker within 5 minutes. All these sites spawn their processes…
Neograph734
  • 143
  • 5
4
votes
3 answers

Scaling issues with Nagios as CaSC

If Nagios server would use a CasC source of configuration in a larger organization is there a design flaw in this approach which will limit scaling up? To avoid broken configuration, test instances of Nagios could be validated in a CI…
Ta Mu
  • 6,792
  • 5
  • 43
  • 83
4
votes
0 answers

Thanos or Cortex - What handles very large scale (say, hundreds of millions of time series) better?

We're looking at a new metrics solution and are attempting to build it in house. So, my question is: What can scale more effectively / with less pain; Thanos or Cortex? I understand the general differences between the two. I'm just looking to…
John Humphreys
  • 1,570
  • 7
  • 18
3
votes
1 answer

Docker memory storage

Docker daemon allows, according to documentatio, using memory storage driver for its images. Has anybody any daily business experiences with that?
Ta Mu
  • 6,792
  • 5
  • 43
  • 83
2
votes
1 answer

Pattern(s) to scale node.js based ETL

I have a node.js ETL application that transforms data from 1..n locations and from each location there will be 0..m transactions. The application is started via cron on a configurable interval, currently 15 minutes. Currently, the pattern employed…
300baud
  • 131
  • 3
1
vote
1 answer

Best method of retrieving environment information via Salt Stack?

I have been tasked with using Salt Stack to find out what files/services exist in each of our environments (Test, UAT, Production). The Salt master can speak to all servers in Test and UAT (Production Salt master is yet to be configured) so I was…
jto
  • 378
  • 1
  • 6
  • 19
1
vote
1 answer

What is it called to deploy major changes for limited number of customers first?

Is there a common name for this deployment strategy which is to avoid deploying major updates across all the customers straight away but rather limit the initial number to see their feedback, find any bugs that slipped through testing thus…
Greendrake
  • 125
  • 1
  • 4
0
votes
1 answer

Scaling a stateful service which has primary/secondary architecture by setting a property

I have a java web stateful service which connects to mongodb and currently it is being deployed as a jar directly on the VM. This service has a property in properties file i.e. primary. If we set primary to true then it is primary instance of the…
SRJ
  • 133
  • 8
0
votes
0 answers

What is the best way to deploy a short living process when you have no machine?

I'm new to devops . I created a converter and want to deploy it . The converter converts a 3D model from a format to another and then you'll be able to visualize the outpout file on the platform and then you can download the file if you want…
tawfikboujeh
  • 9
  • 2