Blackbox testing vs openbox testing applications in a containerized environment

Question

We use containers to deploy our Node.js applications. But currently, the development and a good share of testing is done by the developers on their own host system and thus using different versions of the tools (notably Node.js itself, NPM, and Mocha) depending what's available in the Linux distribution they use.

To standardize our development stack, we would like to perform all tests in containers so everyone will use the exact same version of the tools. The application source directory will be bind-mounted to the "test container".

We are having now a discussion here about how the developers are supposed to interact with the "test container". Basically, using Dockerfile notation, we are hesitating between two options:

• The blackbox solution:

  CMD npm install && npm test
  

  host$ # Running a test suite:
  host$ sudo docker run app-test-container
  

  Running tests do not require any manual work besides starting the container. Tests are guaranteed to be reproducible. But developers complain about the possible overhead of running npm install before every test and a potential issue if (because of the bind mount) they end-up starting concurrently several instances of npm install.

• The openbox solution:

  CMD /bin/bash
  

  host$ # Starting the test container:
  host$ sudo docker run -it app-test-container
  
  app-test-container$ # run tests after dependency change
  app-test-container$ npm install && npm test
  
  app-test-container$ # run tests when no dependency change
  app-test-container$ npm test
  

  With that solution, the developers just gain shell access to a container set up with a standard development stack. It is up to them to npm install when needed and tests are manually started by issuing the npm test command in the container. It is very close to what they currently do on their development host. They also see some advantage in having interactive access to the container so they can start directly node from there for ad-hoc tests and experimentations using the REPL, as well as for directly running other npm commands from the container, basically freeing them to install any tool directly on their host.

---

So, I have several questions:

• Do those solutions seem to be sane to you?
• Can you see some advantages or disadvantages we missed in either solution?
• Overall, is this a good step toward enforcing best practices, or do you spot some anti-pattern here?

... and finally, which solution would you favor?