Strategy for keeping secret info such as API keys out of source control?

Question

I'm working on a website that will allow users to log in using OAuth credentials from the likes of Twitter, Google, etc. To do this, I have to register with these various providers and get a super-secret API key that I have to protect with pledges against various body parts. If my key gets ganked, the part gets yanked.

The API key has to travel with my source, as it is used at runtime to perform authentication requests. In my case, the key must exist within the application in a configuration file or within the code itself. That isn't a problem when I build and publish from a single machine. However, when we throw source control into the mix, things get more complicated.

As I'm a cheap bastard, I'd much prefer to use free source control services such as TFS in the cloud or GitHub. This leaves me with a slight conundrum:

How can I keep my body intact when my API keys are in my code, and my code is available in a public repository?

I can think of a number of ways to handle this, but none of them are that satisfying.

• I could remove all private info from code, and edit it back in after deployment. This would be a severe pain to implement (I won't detail the many ways), and isn't an option.
• I could encrypt it. But as I have to decrypt it, anyone with the source could figure out how to do so. Pointless.
• I could pay for private source control. LOL j/k spend money? Please.
• I could use language features to segregate sensitive info from the rest of my source and therefore keep it from source control. This is what I'm doing now, but it could easily be screwed up by mistakenly checking in the secret file.

I'm really looking for a guaranteed way to ensure I don't share my privates with the world (except on snapchat) that will work smoothly through development, debugging and deployment and be foolproof as well. This is completely unrealistic. So what realistically can I do?

_{Technical details: VS2012, C# 4.5, source control is either going to be TF service or GitHub. Currently using a partial class to split the sensitive keys off in a separate .cs file that won't be added to source control. I think GitHub may have the advantage as .gitignore could be used to ensure that partial class file isn't checked in, but I've screwed that up before. Am hoping for a "oh, common issue, this is how you do it" but I may have to settle for "that doesn't suck as much as it could have", :/}

Answer 1

Don't put your secret information in your code. Put it into a configuration file which is read by your code at startup. Configuration files shouldn't be put on version control, unless they are the "factory defaults", and then they shouldn't have any private information.

See also the question Version control and personal configuration file for how to do this well.

Answer 2

You could put all the private/protected keys as system environment variables. Your configuration file will look like this:

private.key=#{systemEnvironment['PRIVATE_KEY']}

This is how we handle those cases and nothing goes into code. It works very well combined with different property files and profiles. We use different property files for different environments. In our local development environment we put the development keys in the property files to simplify the local setup:

private.key=A_DEVELOPMENT_LONG_KEY

Answer 3

Pure Git way

• .gitignore included file with private data
• Use a local branch, in which you replace TEMPLATE with DATA
• Use smudge/clean filters, in which (local) filter's script perform bidirectional replacement TEMPLATE <-> DATA

Mercurial way

• MQ-patch(es) on top of dummy code, which replace TEMPLATE with DATA (changesets are public, patch is private)
• Keyword extension with specially designed keywords (expanded only in your working directory)

SCM-agnostic way

• Have replacement of keywords as part of build/deploy process

Answer 4

I put secrets into encrypted file(s) which I then commit. The pass phrase is provided when the system launches, or it is stored in small file that I don't commit. It's nice that Emacs will cheerfully manage these encrypted files. For example, emacs init file includes: (load "secrets.el.gpg"), which just works - prompting me for the password on those rare occations when I start the editor. I don't worry about somebody breaking the encryption.

Answer 5

This is very Android/Gradle specific but you could define the keys in your global gradle.properties file located in user home/.gradle/. This is also useful as you can use different properties depending on buildType or flavour i.e API for dev and different one for release.

gradle.properties

MY_PRIVATE_API_KEY=12356abcefg

build.gradle

buildTypes {
        debug{
            buildConfigField("String", "GOOGLE_VERIFICATION_API_KEY", "\"" + MY_PRIVATE_API_KEY +"\"")
            minifyEnabled false
            applicationIdSuffix ".debug"
            }
        }

In code you'd reference like this

String myAPI = BuildConfig.GOOGLE_VERIFICATION_API_KEY;

Answer 6

You're not suppose to distribute that key with your application or store it in the source code repository. This question is asking how to do that, and that isn't what is normally done.

Mobile Web Application

For Android/iPhone the device should request the KEY from your own web service when the app is first run. The key is then stored in a safe location. Should the key be changed or revoked by the publisher. Your web service can publish a new key.

Hosted Web Application

Customers using a license of your software will have to manually input the key when first configuring the software. You can give everyone the same key, different keys or they get their own.

Published Source Code

You store your source code in a public repository but not the KEY. In the configuration of the file you add the lines *place key here*. When a developer uses your source code they make a copy of the sample.cfg file and add their own key.

You do not keep yourconfig.cfg file used for development or production in the repository.

Answer 7

Use environment variables for secret things that change for each server.

http://en.wikipedia.org/wiki/Environment_variable

How to use them is language dependent.

Answer 8

I think this is an issue everyone has had some trouble with at some point.

Here's a workflow I have used, which might work for you. It uses .gitignore with a twist:

1. All configuration files go in a special folder (w/ sample config files - optional)
2. All configuration files are included in .gitignore, so that they don't go public
3. Setup a gitolite server (or your favorite git server) on a private box
4. Add a repo with all the config files in the private server
5. Add a script to copy config files to the special folder in the main repo (optional)

Now, you can clone the config repo to any development and deployment system. Just run the script to copy the files to the correct folder and you're done.

You still get all the GitHub candy, share your code with the world and the sensitive data are never in the main repo, so they don't go public. They are still only a pull and a copy away from any deployment system.

I use a 15$/year box for the private git server, but you can also setup one at home, per the cheapskate requirement ;-)

PS: You could also use a git submodule (http://git-scm.com/docs/git-submodule), but I always forget the commands, so quick & dirty rules!

Answer 9

Use encryption, but provide a master key at startup, as a password at the console, in a file only the process's user can read, or from a system-provided key store like Mac OS keychain or Windows key store.

For continuous delivery, you'll want various keys recorded somewhere. Configuration should be demarcated from code, but it makes a lot of sense to keep it under revision control.

Answer 10

3 Strategies, not yet mentioned(?)

On check in or in a VCS pre-check in hook

• search for strings with high entropy, example- detect-secrets
• regex search for well known API key patterns. AWS's AKIA* keys are an example, git-secrets is one tool based on that. Also, variable names like 'password' with constant assignment.
• search for known secrets- you know your secrets, search text for them. Or use a tool, I wrote this proof of concept.

Strategies already mentioned

• store in file outside of source tree
• have it in source tree, but tell VCS to ignore it
• environment variables are a variation on storing data outside of the source tree
• just don't give the valuable secrets to developers

Answer 11

Keep private information out of your source control. Create a non-loaded default for distribution, and have your VCS ignore the real one. Your installation process (whether manual, configure/build or wizard) should handle creating and populating the new file. Optionally modify permissions on the file to ensure only the required user (webserver?) can read it.

Benefits:

• Doesn't assume development entity == production entity
• Doesn't assume all collaborators/code reviewers are trusted
• Prevent easy mistakes by keeping it out of version control
• Easy to automate installs with custom configuration for QA/builds

If you are already doing this and are accidentally checking it in, add it to your project's .gitignore. This'll make it impossible to do again.

There are plenty of free Git hosts around that provide private repositories. Although you should never version your credentials, you can be cheap and have private repos too. ^_^

Answer 12

Instead of having the OAuth key stored as raw data anywhere, why not run the string through some encryption algorithm, and store it as a salted hash? Then use a configuration file to restore it at runtime. That way the key isn't stored anywhere, whether it is stored on a development box, or the server itself.

You could even create an API such that your server automatically generates a new salted and hashed API key on a per request basis, that way not even your team can see the OAuth source.

Edit: Perhaps try the Stanford Javascript Crypto Library, it allows for some pretty secure symmetric encryption/decryption.