Titanium Ninja

If you enjoy using Ti Inspector for debugging and inspecting the state of your Titanium apps in the iOS simulator, you’ve probably wondered if it would be possible doing the same for apps executed on device. Actually, the way the Ti SDK builds up the connection between the running app and the debugging server implemented in Titanium Studio prevents this to happen in a seamless way when using Ti Inspector. Moreover the client component of the debugger (i.e. the one that is executed within the app) is implemented in a closed source library (libti_ios_debugger.a), making it somewhat hard to create a server component that can seamlessly replace the Titanium Studio debugging server. I must say that I didn’t bother to investigate the issue in too much detail, and I’m working on a different kind of solution involving the use of a custom native module.

Anyway, while the module-based solution is not yet ready, I’d like to share a rather simple way to use Ti Inspector with an app running on an iOS device, even if the required steps might appear a bit convoluted at a first sight. This involves launching the application through Xcode, and using its powerful conditional breakpoints. In particular, Xcode allows setting breakpoints in your application source code, which are triggered only if the specified condition is met. Moreover we can attach specific actions to be executed right after the breakpoint has been hit, as well as enabling the program to resume right away, without interrupting its normal flow.

The most useful among the available actions that can be attached to a breakpoint are “Log Message” and “Debugger Command”. These can be of great help for printing out the state of some variables in our code during its execution, allowing us to add debug log statements at runtime, without even touching the code.

So, without further ado, let’s see how to build our app through Xcode, allowing it to work with Ti Inspector.

First, we need to build the app through Ti Studio or the CLI for running it through the iOS simulator:

$ titanium build -p iphone

Then we need to get the IP address of our development machine on our WiFi network (e.g. en1 interface on a MacBook Pro):

$ ifconfig en1

We’ll get something like:

en1: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    ether xx:xx:xx:xx:xx:xx
    inet 192.168.1.3 netmask 0xffffff00 broadcast 192.168.1.255
    nd6 options=1<PERFORMNUD>
    media: autoselect
    status: active

The current IP address assigned to the WiFi interface in this case is 192.168.1.3

We can now open the Xcode project:

$ open build/iphone/name_of_your_app.xcodeproj

We’ll have to add a couple of breakpoints in TiApp.m, in the -(void)boot method (starting at line 187 in Ti SDK 3.1.3.GA), where the connection with the debugger server is initiated. In particular, we need to fake the presence of a debugger.plist file, then we can set the two host and port parameters where the Ti Inspector debug server is listening, i.e. the IP address we retrieved before (e.g. 192.168.1.3) and for example the default 8999 port.

For setting the required values we use the expr lldb command for evaluating expressions in the debugger, so we edit the two breakpoints adding a “Debugger Command” action:

Add breakpoint and edit

Manipulate the filePath variable

Set the debugger host and port

Once this setup phase is completed we can launch the application, by selecting the appropriate device target.

As a side note, this mechanism also works for simulator targets, without the need to build the app through the CLI with the --debug-host option.

Update (16 Oct 2013)

Matthieu Sieben in a comment points out that instead of using the IP address of your machine you can use the host name followed by .local. This is automatically handled by the Multicast DNS feature of the bonjour protocol.

In order to know the host name of your computer you can use the following command from a terminal:

$ hostname

My Titanium/iOS development workflow mainly revolves around using Sublime Text for code editing, and the Titanium CLI and Xcode for testing/deployment/debugging. However, sometimes this setup isn’t sufficient, especially when I need to inspect what happens in an application at the JavaScript level. In these cases there are basically two choices:

• fill the code with logging statements
• use the JavaScript debugger integrated in Titanium Studio

It turns out that there are times when none of these techniques is satisfying: using logging statements is a quick and dirty technique, but it can lead to long modify/rebuild/test iterations, before finally finding the right spot in the code that’s causing an issue. Using a proper debugger, like the one provided by Ti Studio is surely the way to go in most cases, however, switching back to Studio just for debugging can be a painfully slow operation. In other words, I needed a more “agile” way to start a debugging session.

Enter Ti Inspector

Ti Inspector is a tool I had a lot of fun building over the past few months, which allows debugging a Titanium app (only on iOS for the moment) through the Chrome DevTools debugging interface, i.e. the same panel that can be opened for inspecting and debugging a web page in Google Chrome (e.g. right-click on the page, then Inspect Element).

How was this possible? Actually, the DevTools panel provided by Chrome is nothing more than a pure HTML+CSS+JS web application, whose source code is part of the Blink project, i.e. the fork of the WebKit rendering engine that Google recently started. When opened inside of Chrome, the DevTools front-end directly communicates with the Chrome internals through a series of JS to native bindings, however it can also be used in a remote debugging setup, where the front-end is served from and connects to a remote Chrome instance, running a web application we want to inspect. In particular, the DevTools front-end application expects to communicate with a remote backend counterpart through a websocket connection, implementing a JSON-based RPC protocol, which is documented in detail here.

On the other side we have our Titanium application running (e.g. on the iOS simulator), which integrates a debugger agent that is able to interact with the engine used for executing our JavaScript code (e.g. JavaScriptCore on iOS). If enabled, the debugger agent expects to communicate through a TCP-based protocol with its front-end counterpart, which is normally represented by the Titanium Studio debugger interface. Titanium Studio silently enables this mechanism when we build for debugging, by appending a --debug-host argument to the Titanium CLI invocation, for example:

$ titanium build --platform ios --debug-host localhost:54321

Ti Inspector is the tool that allows these two worlds to successfully communicate, acting as a gateway between the Chrome DevTools remote debugging protocol and the Titanium debugger protocol. It does so by the means of a node.js based application, which implements the following mechanisms:

• It serves the DevTools web app from the default port 8080
• It listens for tcp connections on the default port 8999, where the Ti debugger agent will connect once the app starts
• It accepts websocket connections from the DevTools app
• Once both the debugger agent and DevTools app are connected, Ti Inspector translates commands, replies and asynchronous events from one protocol to the other, doing additional book-keeping and translating the descriptors of the necessary model elements (i.e. scripts, breakpoints, stack frames, variables, etc.).

How to use it

Ti Inspector is a node.js module, so as a basic prerequisite a working node.js setup is needed, then we can use npm for installing it globally:

$ [sudo] npm install -g ti-inspector

Once installed, we can cd to any Titanium Mobile application project directory and launch the ti-inspector command:

$ cd /path/to/your/titanium/project/directory
$ ti-inspector

doing so, a web server starts listening on port 8080, and a debug server is attached to TCP port 8999. Pointing a browser on localhost:8080 we’ll get a page with a brief description of our application, telling that no active debug sessions are active. At this point, we can start our application through the Titanium CLI, specifying that the debug agent running in the app will need to connect to port 8999 on the localhost, for example:

$ titanium build -p iphone --tall --retina --debug-host localhost:8999

Once the app will start in the iOS Simulator, the debugger will connect with Ti Inspector and a new debugging session will be created. In the browser we can then start the DevTools app and start debugging.

Anyway, sometimes a screencast is better then thousand words, so you can check out this short demo:

Features

Ti Inspector supports almost all the JavaScript debugging features made available by the “Sources” panel of the Chrome DevTools interface except the features that are tied specifically to the web environment (e.g. DOM, XHR, EventListener breakpoints and WebWorkers sub-panels). In particular it supports the following:

• Breakpoints: setting/removing breakpoints, conditional breakpoints
• Call stack inspection (when execution is suspended)
• Variables and objects inspection
• Watch expressions
• Step operations (step over, step-into, step-out)
• Console logging
• Expression evaluation in the console (only when execution is suspended)
• Suspend on exceptions (disabled by default)

Limitations

Ti Inspector is currently at an alpha stage of development. Some features are still missing and will be possibly added as they become indispensable (e.g. Android emulator support), while others will probably never taken into consideration (e.g. on device debugging).

For completeness, some of the current limitations are the following:

• Android is not currently supported: for debugging Android Apps, Titanium Studio does more heavy lifting and the Ti debugger protocol is somewhat translated into the V8 debugging protocol by an internal component. This means that supporting Android will mean implementing the V8 remote debugging protocol in Ti Inspector. This is something I’ll likely work on in the near future
• On device debugging is not supported since it’s treated in a special way by the CLI and Studio.
• Expressions can only be evaluated when the execution is suspended

Source code

The source code is completely available on GitHub under the MIT license. Issue postings and pull requests are very welcome.

Courtesy Tipsy & Tumbler Ltd

Almost two weeks ago I had the pleasure to attend tiConf 2013 in Valencia, where, besides getting in touch with lots of awesome developers from all over the world, I held a workshop on native module development for Titanium on both iOS and Android platforms. Here’s the presentation:

During the talk I gave a quick tour on how to start creating modules, with an emphasis on how JS-to-native bindings are structured in the Titanium framework, presenting a set of real-world use cases. Among the others, I mentioned a module for offloading to native code the conversion of XML documents into JSON structures, which wasn’t yet published.

I had this module lying around for several months, so in the past few days I took some time to polish the iOS version and I published it on GitHub here: github.com/omorandi/TiXml2Json. I think it represents a very good start for understanding some of the concepts around Titanium native module creation.

But why shall we need such kind of functionality implemented in native code in the first place? Actually, parsing XML structures through the DOM API is somewhat cumbersome and inefficient, while working with JSON structures is more easily manageable. For doing this we could use any of the JavaScript modules available, like the good XMLTools from David Bankier, however, when the size of the XML string is relevant, the parsing phase can be very slow, thus blocking the execution of our app for a significant and noticeable amount of time. A possible solution is then to do the conversion in native land, achieving better performance. In some cases this may still not be enough, so I also implemented an asynchronous version of the conversion method, which offloads the conversion process on a separate thread, keeping the JavaScript thread free to continue its normal flow, thus adding responsiveness to the client application.

The module exports a very simple API with just two methods:

• convert(xml)
• convertAsync(xml, callback)

The former takes an xml string or blob argument and returns the corresponding JSON representation as a JS object. The latter implements the asynchronous version described above, which takes an additional argument for the callback function that will be invoked with the resulting JS object at the end of the process. Internally, the conversion methods use the XML to NSDictionary converter code from Troy Brant.

This week I also published another iOS module: TiAssetsLibrary, which wraps the iOS AssetsLibrary framework with an almost 1 to 1 mapping between the JS and the native API. This module deserves another post where I’d like to talk about performance implications of API design.

That’s all for now. See you all at the next tiConf.

Around four months ago I started experimenting with some custom modifications to the JavaScriptCore library, which is the JavaScript engine used by the Titanium Mobile SDK for executing Titanium applications on iOS. After succeeding in enabling the profiler component provided by JSCore and extracting some relevant profiling data from running applications, I wanted a tool that would allow me to show such information in a manageable way, like the Web Inspector and Developer tools allow to do in Safari or Chrome.

My basic requirements were the following:

• The tool needed to be able to start and stop the profiler running alongside a Titanium Mobile application executed on the iOS Simulator, as well as to retrieve the collected profiling data
• The GUI app should be very simple and quick and easy to develop: a web app would be good enough
• I wanted to be able to show the profiling data (i.e. a function call tree showing the amount of time spent in each function) in the typical expandable tree grid
• I wanted to be able to quickly show the source code of every referenced function, in order to better understand the data every time an anonymous function was reported

So I started creating a very simple web-based GUI and a node.js server component providing a proxy between the client app and the profiler, and today I’m able to introduce some results, with the following demo screencast:

This has been possible thanks to the following technologies/libraries:

• Ext JS: General GUI layout & tree-grid
• ACE editor: for showing the original source code of the selected application files
• socket.io: for event-based communication between all the components

The screencast showcases the usage of the three main components involved:

• The node-based server, used for serving the client app files and for communicating with both the profiler and the GUI
• The GUI app itself
• The Titanium Mobile application, built and executed through a custom version of the 2.1.0.GA Titanium SDK, with all the required additions to make the magic possible

The node server gets notified when the Titanium application and the profiler are started, re-broadcasting such events. The web-app, on the other side, issues start & stop commands to the node server, which reflects them to the profiler running inside of the Ti app.

At the core of the system lies a custom version of the TiJSCore library, and some very limited additions to the inner workings of the Titanium Kroll bridge. In particular, besides the basic modifications already mentioned in the previous post, it has been necessary to do the following:

• modifying the code managing the CommonJS require() function, in order to make both the JS interpreter and the profiler aware of the names of the files being loaded through it
• decorating the calls to methods of Titanium proxy objects with a representation of their real name, in order to highlight them with something more meaningful than (Function oject). For example a call to Ti.UI.Window.open() now it’s reported as [object TiUIWindow].open: not exactly the JavaScript signature, but quite similar, indeed
• building a bit of infrastructure in order to allow the profiler to communicate remotely, either for receiving start and stop commands, either for sending the recorded profiling data. Using the iOS port of the socket.io library helped a lot in this context.
• other minor adjustments (e.g. converting profiling data returned by the JSCore profiler to JSon)

At the moment I’m not planning to release these modifications and additions as open source. I’m still thinking about what to do about them. Actually it would be nice if at some point they could be integrated into the Titanium SDK.

Anyway, I’m willing to share this tool with the Titanium community, letting interested folks to play with it, and report any kind of feedback, so I’ve created a repository on github hosting both the node server and the client app source code.

Originally I was aiming to keep all the modifications to the JSCore library, as well as the corresponding hooks inside the Titanium SDK completely self-contained in a single binary, in order to be able to provide just a simple drop-in replacement for the libTiCore.a library, to be plugged in a standard distribution of the Ti SDK. However, after integrating socket.io on the iOS side, things are a bit more complicated, since socket.io requires the target app to be linked with the Security framework, which is not linked by default in the standard Titanium Xcode template. For this reason, in order to keep things super-simple for anyone interested in this project, I’ve prepared a package containing a custom version of the 2.1.0.GA Titanium SDK that can coexist side by side with the official SDKs provided by Appcelerator already present in the system.

Anybody wanting to follow all the steps, without using the prebuilt package, can download a custom build of the libTiCore.a library, copy it in the ${TI_SDK}/iphone directory of a 2.X.X version of the Ti SDK (where ${TI_SDK} for example is /Library/Application\ Support/Titanium/mobilesdk/osx/2.1.0.GA, open the ${TI_SDK}/iphone/iphone/Titanium.xcodeproj) and add there a reference to the Security framework.

The custom SDK can be used for building, executing and profiling any Titanium application on iOS, with the following, and maybe more, limitations:

• It doesn’t work for Android targets
• It can be used only with apps running inside the iOS Simulator: apps should be able to execute on device, but at the moment, no profiling data can be exchanged with the node.js server component. Results of building and running Ti apps on device with such SDK are currently undefined
• The client app is currently very limited

Wrap up

Where do we go from here?

I have a bunch of things I’d like to improve/work on in the very near future. Here are some:

• Replace the currently very limited web-app with the client from the Node Inspector project. This is actually a fork of the WebKit Web Inspector that can be used to debug and experimentally profile node.js applications. Modifying this to work with my server should be fairly easy
• Improve the way in which anonymous functions are reported. It is true that these functions are actually unnamed, but in most cases they’re usually referenced through a variable or an object member. I’d like to leverage the features of the super-cool Esprima JavaScript parser in order to extract such information directly from source files, reporting the name of the referencing variable/member when possible
• Make things work on device
• Start studying how to do something similar on V8 for android targets

Any comment is well accepted, either here, or on twitter. This project is needing some love, especially on the GUI component and node server side. Feel free to fork it on github and make pull requests.

As the code base of an application grows, manual testing becomes very quickly a cumbersome and highly error prone activity. Moreover, when fixing bugs or refactoring portions of the app, there’s no way for telling if a fix is breaking something that was previously working. In order to mitigate such drawbacks, automatic unit testing tools come at help, allowing to describe how each single component of an application must behave, and testing in a systematic way if the imposed requirements and expectations are met.

During the last year, which I mainly spent developing and maintaining a quite large and complex Ti Mobile application, I tried to investigate which were the available options for gradually introducing automatic testing in my development workflow. There is plenty of tools and frameworks for Javascript testing, and although most of them are obviously targeted to the web development world, some of them have been more or less successfully made available for testing Titanium Mobile applications.

JS dynamic nature makes it an ideal language for automatic testing: you can easily modify objects at runtime, and provide fake implementations around the code that needs to be tested. So testing JS applications, and in particular Ti Mobile applications is super easy, right? Wrong! Or better, it can be, provided that you use the right tools and that you write your code in a way that makes it easily testable. For instance, every time we use state variables and functions hidden inside closures (e.g. even just using either CommonJS, or the omni-present module pattern), we are creating code that can be hard to test. An interesting take on this problem can be found here.

In this short series of post I’d like to talk about some tools and patterns I’ve found useful for introducing automatic testing in my coding/refactoring habits.

The tools

There are quite a lot JavaScript testing frameworks available, some of which are dependent on the presence of a DOM, so they’re only suitable for in-browser testing and cannot be used in Titanium. Among those that are framework-independent I’ve found Jasmine.js very interesting. Jasmine is a self-contained Behaviour-Driven Development (BDD) framework for Javascript, where tests express the “expected behaviour” for our code. I won’t go into detail on the philosophy behind this kind of Test-Driven Development practice, but those willing to dig deeper will find the “RSpec Book” quite useful.

There exist a bunch of attempts bring BDD testing with Jasmine to Titanium:

• akahigeg Jasmine-Titanium (github)
• guilhermechapiewski Titanium-Jasmine (github)
• Cross-platform, Testable Mobile Goodness with Titanium and Jasmine
• probably others more recents…

I have tried each of these approaches and none satisfied me. In particular, the second solution, which is probably the most used among the Ti community and is based on running tests directly in the target simulator, dissatisfied me for the following reasons:

• Testing in the simulator is very slow, while TDD is based on quick red/green/refactor iterations, where you first write the test for your code, you make it fail, you refactor and iterate until you get a green light. Even if, like me, you don’t follow religiously this practice, you quickly lose your patience waiting for the simulator to start every time
• I think that testing in the simulator, while surely necessary at some point (mainly while doing integration and acceptance testing), allows taking short paths that quickly lead to writing poorly testable code (more on this later)

Actually what I was looking for was a solution allowing me to write and execute my tests in the quickest and simplest way possible. Enter Jasmine-Node, which is a Node.js module, providing a cli for running Jasmine tests through node.

Jasmine-node: setup & execution

Supposing you already have Node.js and npm in place, installing jasmine-node is just a matter of typing:

$ sudo npm install jasmine-node -g

Now you can start executing your tests (also called specs), whose JS files should reside all in the same directory tree. For example, let’s consider the following directory tree, which could be part of a Ti Mobile project:

project/
|
+---- Resources/
|     |
|     +---- app.js
|     |
|     +---- app-modules/
|           |
|           +---- backend.js
|           |
|           +---- ui.js
|
+---- spec/
      |
      +---- backend_spec.js
      |
      +---- ui_spec.js

In Resources we have the application code, contained in a couple of CommonJS modules. Our tests reside in the spec directory. Please note that each file in this directory contain the spec string in its name, allowing jasmine-node to recognize it as a file to be evaluated by the test runner.

Having in mind this directory structure, we can execute our tests (e.g. from inside the project dir):

$ jasmine-node spec
.............

Finished in 0.013 seconds
13 tests, 15 assertions, 0 failures

the command argument is simply the path of the directory containing our tests.

Let’s write some code and some tests

Now, we can write our code and our tests. For example, suppose in our app we have a module with some silly utility functions:

1
2
3

exports.computeSum = function(a, b) {
  return a + b;
};

we can write our Jasmine spec like this:

1
2
3
4
5
6
7
8
9

var util = require('../Resources/app-modules/util');
describe('util tests', function() {

  it('should compute the sum between 1 & 2', function(){
      var sum = util.computeSum(1, 2);
      expect(sum).toEqual(3);
  });

});

let’s run the spec:

jasmine-node spec
.

Finished in 0.007 seconds
1 test, 1 assertion, 0 failures

I don’t enter in further details on how to write Jasmine tests, as this will be covered in the second part of this post (still to come);

We need to remember that specs must be written as Node.js modules and obviously the Titanium Mobile API objects won’t be available. So how can we test our Titanium modules if they make use of the Titanium API? We need to provide our tests with fake implementations of it. This subject leads to the concept of mocks and stubs and dependency injection, which I’ll cover in the next episode.

Caveats

In this post we’ve just touched the basics about how to setup our environment for testing with node-jasmine and Titanium, however there are some somewhat hidden issues that need to be taken into account. In particular, it turns out that the Titanium implementation of CommonJS require() is buggy and doesn’t correctly support relative paths. This represents a major problem when trying to integrate jasmine-node test runners in projects with even minimally complex directory trees.

A possible solution to the issue is to not use relative paths in require() in Titanium (but you are free to use them in your jasmine specs run through node). Instead of relative paths we need to use full paths with Resources as the root directory.

For example, taking into account the directory tree laid out before, let’s suppose the backend.js module is dependent from the util.js module, we’ll need to write the require() in this way:

1
2
3

require('app-modules/util');

//...

In our jasmine specs we’ll use relative paths as usual.

Now the trick is to set the NODE_PATH environment variable to point to the Resources directory of the Ti Project, just like:

export NODE_PATH="${PROJECT_DIR}/Resources"
jasmine-node spec

This will instruct the Node.js environment to search for modules in the Resources directory, allowing them to be found by the test runner.

These two lines can obviously be encapsulated in a shell script or makefile.

What next?

In the next episode of this series, we’ll see how to write our application modules in Titanium, in order to make them easily testable. We’ll also expand our use of available testing frameworks, discussing how to use Sinon.js for creating mocks and stubs.