Skip to content

MASTG-TEST-0084: Testing for Debugging Code and Verbose Error Logging

Overview

Static Analysis

You can take the following static analysis approach for the logging statements:

  1. Import the application's code into Xcode.
  2. Search the code for the following printing functions: NSLog, println, print, dump, debugPrint.
  3. When you find one of them, determine whether the developers used a wrapping function around the logging function for better mark up of the statements to be logged; if so, add that function to your search.
  4. For every result of steps 2 and 3, determine whether macros or debug-state related guards have been set to turn the logging off in the release build. Please note the change in how Objective-C can use preprocessor macros:
#ifdef DEBUG
    // Debug-only code
#endif

The procedure for enabling this behavior in Swift has changed: you need to either set environment variables in your scheme or set them as custom flags in the target's build settings. Please note that the following functions (which allow you to determine whether the app was built in the Swift 2.1. release-configuration) aren't recommended, as Xcode 8 and Swift 3 don't support these functions:

  • _isDebugAssertConfiguration
  • _isReleaseAssertConfiguration
  • _isFastAssertConfiguration.

Depending on the application's setup, there may be more logging functions. For example, when CocoaLumberjack is used, static analysis is a bit different.

For the "debug-management" code (which is built-in): inspect the storyboards to see whether there are any flows and/or view-controllers that provide functionality different from the functionality the application should support. This functionality can be anything from debug views to printed error messages, from custom stub-response configurations to logs written to files on the application's file system or a remote server.

As a developer, incorporating debug statements into your application's debug version should not be a problem as long as you make sure that the debug statements are never present in the application's release version.

In Objective-C, developers can use preprocessor macros to filter out debug code:

#ifdef DEBUG
    // Debug-only code
#endif

In Swift 2 (with Xcode 7), you have to set custom compiler flags for every target, and compiler flags have to start with "-D". So you can use the following annotations when the debug flag DMSTG-DEBUG is set:

#if MSTG_DEBUG
    // Debug-only code
#endif

In Swift 3 (with Xcode 8), you can set Active Compilation Conditions in Build settings/Swift compiler - Custom flags. Instead of a preprocessor, Swift 3 uses conditional compilation blocks based on the defined conditions:

#if DEBUG_LOGGING
    // Debug-only code
#endif

Dynamic Analysis

Dynamic analysis should be executed on both a simulator and a device because developers sometimes use target-based functions (instead of functions based on a release/debug-mode) to execute the debugging code.

  1. Run the application on a simulator and check for output in the console during the app's execution.
  2. Attach a device to your Mac, run the application on the device via Xcode, and check for output in the console during the app's execution.

For the other "manager-based" debug code: click through the application on both a simulator and a device to see if you can find any functionality that allows an app's profiles to be pre-set, allows the actual server to be selected or allows responses from the API to be selected.