How to avoid a regular user can only test in production?

Question

When configuring test environments, the issue often comes up where I ask the question to the customer like in my question title here:

How to avoid a regular user can only test in production?

There are typically the environments like unit test, system test, integration test, volume test, user acceptance test, etc (I bet there are yet another dozen of names like those). But they all relate to environments in which something differs from how things currently work in the real production environments. So none of these is what my question is about.

To make this a less abstract question, consider this sample below:

Can non-SE employees only test SE sites in production?

I run into this scenario over and over (earlier today it happened again ...):

• I discover some facility on an SE site that gets my attention (that I want to start using).
• I start looking around in help pages, etc, and either don't find what I'm looking for, or what I find is not 100 accurate or clear to me.
• I just try to start to use it, see what happens, hoping I will find the answer to what I couldn't find in the help pages (by just using that facility in some SE site ... read: by testing in production).
• I then just wait and see what happens, which may be things like:
  • I learn about something new/interesting and it works great.
  • I get accused of trying to "fuzzle the system".
  • I get corrected by a mod because of something I did (which I shouldn't have done).
  • I discover a bug, which I either report (on meta.SE), or which I decide to consider as "a bug becomes a feature" (read: I don't want to disclose this handy feature).

However, doing some of those "experiments in production" may damage your reputation (not really your "points", but your real reputation).

So as a good SE citizens, I would be happy to do those experiments in an environment that is designed just for that: a none-production environment designed for users like me who want to do such experiments.

Voilà, if you were asked to configure test environments for SE sites, what would you do (or recommend SE management) about this DevOps issue?

PS: to my knowledge, at this very moment, there is no such SE environment that I can use right now to go check if a user that I "invite right now" for DevOps (while it is in private beta), and which I know did NOT commit to DevOps, will actually be able to login to the DevOps site. So instead of discussing it with mods on some other site, I already ... tested it in production. And despite what some moderator try to make me believe ... it did work (and the invited user now also has an account on DevOps.SE).

Answer 1

A Sandbox could be part of the solution

To bring material to the subject you can check the questions with tag sandbox on MetaSE, there are sanboxes for:

• Q/A formatting.
• Comments formatting.
• Chat.

There's a feature request about "Can we get a Stack Exchange sandbox?" which has no status (neither declined, pending, review nor in-progress meta-tag).

So I assume the answer (to the specific sample in the question) is that there's no way to test out of real SE sites for now and my recommendation to SE would be to accept this feature request.

When is a sandbox appropriate?

I don't think it's always needed to give users a sandbox, and it can be overkill sometimes. Defects should be spotted on Q/A environments, and when they're not, well, either you can know there's a glitch by a fine monitoring of your logs, or you will just never know until someone tells you.

Sanboxes come at an additional cost

Even if you deploy a sandbox for your users to try and confirm a bug, nothing tells you they will inform you of a bug/problem and you had to go through adding a public environment with dummy or anonymised data with no idea of it's ROI.

So It boils down to: "You can create a sandbox for your users but it adds complexity to your infrastructure for a hardly measured return"

Recommendation

My personal opinion would be to put the efforts in log analysis to spot unwanted behaviors instead.