MASTG-TEST-0011: Testing Memory for Sensitive Data
Overview¶
Analyzing memory can help developers identify the root causes of several problems, such as application crashes. However, it can also be used to access sensitive data. This section describes how to check for data disclosure via process memory.
First identify sensitive information that is stored in memory. Sensitive assets have likely been loaded into memory at some point. The objective is to verify that this information is exposed as briefly as possible.
To investigate an application's memory, you must first create a memory dump. You can also analyze the memory in real-time, e.g., via a debugger. Regardless of your approach, memory dumping is a very error-prone process in terms of verification because each dump contains the output of executed functions. You may miss executing critical scenarios. In addition, overlooking data during analysis is probable unless you know the data's footprint (either the exact value or the data format). For example, if the app encrypts with a randomly generated symmetric key, you likely won't be able to spot it in memory unless you can recognize the key's value in another context.
Therefore, you are better off starting with static analysis.
Static Analysis¶
When performing static analysis to identify sensitive data that is exposed in memory, you should:
- Try to identify application components and map where data is used.
- Make sure that sensitive data is handled by as few components as possible.
- Make sure that object references are properly removed once the object containing the sensitive data is no longer needed.
- Make sure that garbage collection is requested after references have been removed.
- Make sure that sensitive data gets overwritten as soon as it is no longer needed.
- Don't represent such data with immutable data types (such as
StringandBigInteger). - Avoid non-primitive data types (such as
StringBuilder). - Overwrite references before removing them, outside the
finalizemethod. - Pay attention to third-party components (libraries and frameworks). Public APIs are good indicators. Determine whether the public API handles the sensitive data as described in this chapter.
- Don't represent such data with immutable data types (such as
The following section describes pitfalls of data leakage in memory and best practices for avoiding them.
Don't use immutable structures (e.g., String and BigInteger) to represent secrets. Nullifying these structures will be ineffective: the garbage collector may collect them, but they may remain on the heap after garbage collection. Nevertheless, you should ask for garbage collection after every critical operation (e.g., encryption, parsing server responses that contain sensitive information). When copies of the information have not been properly cleaned (as explained below), your request will help reduce the length of time for which these copies are available in memory.
To properly clean sensitive information from memory, store it in primitive data types, such as byte-arrays (byte[]) and char-arrays (char[]). You should avoid storing the information in mutable non-primitive data types.
Make sure to overwrite the content of the critical object once the object is no longer needed. Overwriting the content with zeroes is one simple and very popular method:
Example in Java:
byte[] secret = null;
try{
//get or generate the secret, do work with it, make sure you make no local copies
} finally {
if (null != secret) {
Arrays.fill(secret, (byte) 0);
}
}
Example in Kotlin:
val secret: ByteArray? = null
try {
//get or generate the secret, do work with it, make sure you make no local copies
} finally {
if (null != secret) {
Arrays.fill(secret, 0.toByte())
}
}
This doesn't, however, guarantee that the content will be overwritten at runtime. To optimize the bytecode, the compiler will analyze and decide not to overwrite data because it will not be used afterwards (i.e., it is an unnecessary operation). Even if the code is in the compiled DEX, the optimization may occur during the just-in-time or ahead-of-time compilation in the VM.
There is no silver bullet for this problem because different solutions have different consequences. For example, you may perform additional calculations (e.g., XOR the data into a dummy buffer), but you'll have no way to know the extent of the compiler's optimization analysis. On the other hand, using the overwritten data outside the compiler's scope (e.g., serializing it in a temp file) guarantees that it will be overwritten but obviously impacts performance and maintenance.
Then, using Arrays.fill to overwrite the data is a bad idea because the method is an obvious hooking target (see Method Hooking for more details).
The final issue with the above example is that the content was overwritten with zeroes only. You should try to overwrite critical objects with random data or content from non-critical objects. This will make it really difficult to construct scanners that can identify sensitive data on the basis of its management.
Below is an improved version of the previous example:
Example in Java:
byte[] nonSecret = somePublicString.getBytes("ISO-8859-1");
byte[] secret = null;
try{
//get or generate the secret, do work with it, make sure you make no local copies
} finally {
if (null != secret) {
for (int i = 0; i < secret.length; i++) {
secret[i] = nonSecret[i % nonSecret.length];
}
FileOutputStream out = new FileOutputStream("/dev/null");
out.write(secret);
out.flush();
out.close();
}
}
Example in Kotlin:
val nonSecret: ByteArray = somePublicString.getBytes("ISO-8859-1")
val secret: ByteArray? = null
try {
//get or generate the secret, do work with it, make sure you make no local copies
} finally {
if (null != secret) {
for (i in secret.indices) {
secret[i] = nonSecret[i % nonSecret.size]
}
val out = FileOutputStream("/dev/null")
out.write(secret)
out.flush()
out.close()
}
}
For more information, take a look at Securely Storing Sensitive Data in RAM.
In the "Static Analysis" section, we mentioned the proper way to handle cryptographic keys when you are using AndroidKeyStore or SecretKey.
For a better implementation of SecretKey, look at the SecureSecretKey class below. Although the implementation is probably missing some boilerplate code that would make the class compatible with SecretKey, it addresses the main security concerns:
- No cross-context handling of sensitive data. Each copy of the key can be cleared from within the scope in which it was created.
- The local copy is cleared according to the recommendations given above.
Example in Java:
public class SecureSecretKey implements javax.crypto.SecretKey, Destroyable {
private byte[] key;
private final String algorithm;
/** Constructs SecureSecretKey instance out of a copy of the provided key bytes.
* The caller is responsible of clearing the key array provided as input.
* The internal copy of the key can be cleared by calling the destroy() method.
*/
public SecureSecretKey(final byte[] key, final String algorithm) {
this.key = key.clone();
this.algorithm = algorithm;
}
public String getAlgorithm() {
return this.algorithm;
}
public String getFormat() {
return "RAW";
}
/** Returns a copy of the key.
* Make sure to clear the returned byte array when no longer needed.
*/
public byte[] getEncoded() {
if(null == key){
throw new NullPointerException();
}
return key.clone();
}
/** Overwrites the key with dummy data to ensure this copy is no longer present in memory.*/
public void destroy() {
if (isDestroyed()) {
return;
}
byte[] nonSecret = new String("RuntimeException").getBytes("ISO-8859-1");
for (int i = 0; i < key.length; i++) {
key[i] = nonSecret[i % nonSecret.length];
}
FileOutputStream out = new FileOutputStream("/dev/null");
out.write(key);
out.flush();
out.close();
this.key = null;
System.gc();
}
public boolean isDestroyed() {
return key == null;
}
}
Example in Kotlin:
class SecureSecretKey(key: ByteArray, algorithm: String) : SecretKey, Destroyable {
private var key: ByteArray?
private val algorithm: String
override fun getAlgorithm(): String {
return algorithm
}
override fun getFormat(): String {
return "RAW"
}
/** Returns a copy of the key.
* Make sure to clear the returned byte array when no longer needed.
*/
override fun getEncoded(): ByteArray {
if (null == key) {
throw NullPointerException()
}
return key!!.clone()
}
/** Overwrites the key with dummy data to ensure this copy is no longer present in memory. */
override fun destroy() {
if (isDestroyed) {
return
}
val nonSecret: ByteArray = String("RuntimeException").toByteArray(charset("ISO-8859-1"))
for (i in key!!.indices) {
key!![i] = nonSecret[i % nonSecret.size]
}
val out = FileOutputStream("/dev/null")
out.write(key)
out.flush()
out.close()
key = null
System.gc()
}
override fun isDestroyed(): Boolean {
return key == null
}
/** Constructs SecureSecretKey instance out of a copy of the provided key bytes.
* The caller is responsible of clearing the key array provided as input.
* The internal copy of the key can be cleared by calling the destroy() method.
*/
init {
this.key = key.clone()
this.algorithm = algorithm
}
}
Secure user-provided data is the final secure information type usually found in memory. This is often managed by implementing a custom input method, for which you should follow the recommendations given here. However, Android allows information to be partially erased from EditText buffers via a custom Editable.Factory.
EditText editText = ...; // point your variable to your EditText instance
EditText.setEditableFactory(new Editable.Factory() {
public Editable newEditable(CharSequence source) {
... // return a new instance of a secure implementation of Editable.
}
});
Refer to the SecureSecretKey example above for an example Editable implementation. Note that you will be able to securely handle all copies made by editText.getText if you provide your factory. You can also try to overwrite the internal EditText buffer by calling editText.setText, but there is no guarantee that the buffer will not have been copied already. If you choose to rely on the default input method and EditText, you will have no control over the keyboard or other components that are used. Therefore, you should use this approach for semi-confidential information only.
In all cases, make sure that sensitive data in memory is cleared when a user signs out of the application. Finally, make sure that highly sensitive information is cleared out the moment an Activity or Fragment's onPause event is triggered.
Note that this might mean that a user has to re-authenticate every time the application resumes.
Dynamic Analysis¶
Static analysis will help you identify potential problems, but it can't provide statistics about how long data has been exposed in memory, nor can it help you identify problems in closed-source dependencies. This is where dynamic analysis comes into play.
There are various ways to analyze the memory of a process, e.g. live analysis via a debugger/dynamic instrumentation and analyzing one or more memory dumps.
Retrieving and Analyzing a Memory Dump¶
Whether you are using a rooted or a non-rooted device, you can dump the app's process memory with objection and Fridump. You can find a detailed explanation of this process in Process Exploration, in the chapter "Tampering and Reverse Engineering on Android".
After the memory has been dumped (e.g. to a file called "memory"), depending on the nature of the data you're looking for, you'll need a set of different tools to process and analyze that memory dump. For instance, if you're focusing on strings, it might be sufficient for you to execute the command strings or rabin2 -zz to extract those strings.
# using strings
$ strings memory > strings.txt
# using rabin2
$ rabin2 -ZZ memory > strings.txt
Open strings.txt in your favorite editor and dig through it to identify sensitive information.
However if you'd like to inspect other kind of data, you'd rather want to use radare2 and its search capabilities. See radare2's help on the search command (/?) for more information and a list of options. The following shows only a subset of them:
$ r2 <name_of_your_dump_file>
[0x00000000]> /?
Usage: /[!bf] [arg] Search stuff (see 'e??search' for options)
|Use io.va for searching in non virtual addressing spaces
| / foo\x00 search for string 'foo\0'
| /c[ar] search for crypto materials
| /e /E.F/i match regular expression
| /i foo search for string 'foo' ignoring case
| /m[?][ebm] magicfile search for magic, filesystems or binary headers
| /v[1248] value look for an `cfg.bigendian` 32bit value
| /w foo search for wide string 'f\0o\0o\0'
| /x ff0033 search for hex string
| /z min max search for strings of given size
...
Runtime Memory Analysis¶
Instead of dumping the memory to your host computer, you can alternatively use r2frida. With it, you can analyze and inspect the app's memory while it's running.
For example, you may run the previous search commands from r2frida and search the memory for a string, hexadecimal values, etc. When doing so, remember to prepend the search command (and any other r2frida specific commands) with a backslash : after starting the session with r2 frida://usb//<name_of_your_app>.
For more information, options and approaches, please refer to Process Exploration for more information.
Explicitly Dumping and Analyzing the Java Heap¶
For rudimentary analysis, you can use Android Studio's built-in tools. They are on the Android Monitor tab. To dump memory, select the device and app you want to analyze and click Dump Java Heap. This will create a .hprof file in the captures directory, which is on the app's project path.
To navigate through class instances that were saved in the memory dump, select the Package Tree View in the tab showing the .hprof file.
For more advanced analysis of the memory dump, use the Eclipse Memory Analyzer Tool (MAT). It is available as an Eclipse plugin and as a standalone application.
To analyze the dump in MAT, use the hprof-conv platform tool, which comes with the Android SDK.
./hprof-conv memory.hprof memory-mat.hprof
MAT provides several tools for analyzing the memory dump. For example, the Histogram provides an estimate of the number of objects that have been captured from a given type, and the Thread Overview shows processes' threads and stack frames. The Dominator Tree provides information about keep-alive dependencies between objects. You can use regular expressions to filter the results these tools provide.
Object Query Language studio is a MAT feature that allows you to query objects from the memory dump with an SQL-like language. The tool allows you to transform simple objects by invoking Java methods on them, and it provides an API for building sophisticated tools on top of the MAT.
SELECT * FROM java.lang.String
In the example above, all String objects present in the memory dump will be selected. The results will include the object's class, memory address, value, and retain count. To filter this information and see only the value of each string, use the following code:
SELECT toString(object) FROM java.lang.String object
Or
SELECT object.toString() FROM java.lang.String object
SQL supports primitive data types as well, so you can do something like the following to access the content of all char arrays:
SELECT toString(arr) FROM char[] arr
Don't be surprised if you get results that are similar to the previous results; after all, String and other Java data types are just wrappers around primitive data types. Now let's filter the results. The following sample code will select all byte arrays that contain the ASN.1 OID of an RSA key. This doesn't imply that a given byte array actually contains an RSA (the same byte sequence may be part of something else), but this is probable.
SELECT * FROM byte[] b WHERE toString(b).matches(".*1\.2\.840\.113549\.1\.1\.1.*")
Finally, you don't have to select whole objects. Consider an SQL analogy: classes are tables, objects are rows, and fields are columns. If you want to find all objects that have a "password" field, you can do something like the following:
SELECT password FROM ".*" WHERE (null != password)
During your analysis, search for:
- Indicative field names: "password", "pass", "pin", "secret", "private", etc.
- Indicative patterns (e.g., RSA footprints) in strings, char arrays, byte arrays, etc.
- Known secrets (e.g., a credit card number that you've entered or an authentication token provided by the backend)
- etc.
Repeating tests and memory dumps will help you obtain statistics about the length of data exposure. Furthermore, observing the way a particular memory segment (e.g., a byte array) changes may lead you to some otherwise unrecognizable sensitive data (more on this in the "Remediation" section below).