Custom Android Resource Files

Android resource files are a great way to keep your data separate from your code. It works great as long as all you need is one of the supported types. What do you do if you have a more complex static data type that you’d like to manage independently of the code?

The answer is to create a ‘raw’ XML file under the /res/xml directory and write the parser logic yourself. For example, lets say you want to compare the elevation of the mountains in your state. As mountains tend to stick around a while there is no reason to query a database every time you want an elevation. Instead, you can simply create a static XML file like so:

/res/xml/mountain_data.xml

<?xml version="1.0" encoding="utf-8"?>
<mountains>
    <mountain
        name="Mt. Elbert"
        county="Lake"
        elevation="14433"
        lat="39.11807"
        long="106.445417"
        range="Sawatch"
        rank="1" />
    <mountain
        name="Mt. Massive"
        county="Lake"
        elevation="14421"
        lat="39.187298"
        long="106.475548"
        range="Sawatch"
        rank="2" />
    <mountain
        name="Mt. Harvard"
        county="Chaffee"
        elevation="14420"
        lat="38.924328"
        long="106.320618"
        range="Sawatch"
        rank="3" />
</mountains>

Custom resource types don’t get fully precompiled into the ‘R’ class, and hence, you need to load and parse them yourself. At runtime simply load the file using
Resources.getXML(R.fileId) and then parse the data using XmlResourceParser. This parser is very basic and steps through each element every time you call
next(). With each element you can call getEventType() to determine if its a close or open tag. The following code will load, parse and store the elevation
of each mountain in the resource file into a list.

        List&amp;lt;String&amp;gt; elevations = new ArrayList&amp;lt;String&amp;gt;();
        Resources res = getResources();
        XmlResourceParser xrp = res.getXml(R.xml.mountain_data);
        try{
            xrp.next(); // skip first 'mountains' element
            while (xrp.getEventType() != XmlResourceParser.END_DOCUMENT) {
                xrp.next(); // get first 'mountain' element
                if(xrp.getEventType() == XmlResourceParser.START_TAG) {
                    // double check its the right element
                    if(xrp.getName().equals(&amp;quot;mountain&amp;quot;)) {
                        // extract the data you want
                        int count = xrp.getAttributeCount();
                        String name = xrp.getAttributeValue(null, &amp;quot;name&amp;quot;);
                        String elev = xrp.getAttributeValue(null, &amp;quot;elevation&amp;quot;);
                        
                        // add elevation to the list
                        elevations.add(elev);
                        
                        Log.v(TAG, &amp;quot;Attribute Count &amp;quot; + count);
                        Log.v(TAG, &amp;quot;Peak Name &amp;quot; + name);
                    }
                }
            }
        } catch (Exception e) {
            Log.w(TAG, e.toString());
        } finally {
            xrp.close();
            Log.i(TAG, elevations.toString());
        }

Doing this allows you to shrink or expanded the number of mountains without impacting or touching any code.

Android App Collection Widgets

Adding a “launcher” widget to your application can be a little convoluted even in recent 4.x AOSP releases. If you’re heading down this path check out Google’s App Widget tutorial first. Take note that one of the more powerful concepts covered in the tutorial is “collections” or RemoteViews. I created the following UML static structure to help aid in designing your new collection widget.

appwidgetproviderwatermark

Creative Altruism

“Every man must decide whether he will walk in the light of creative altruism or in the darkness of destructive selfishness”

Martin Luther King Jr.

Sniffing Android Emulator

The emulator shipped with ADT is a great tool for both early stage framework development and late stage application development. A large majority of the Android emulator users will only use a small portion of the features for app development. For the developers working on distributed framework development it is a necessity to be able to sniff emulator network traffic (no Wireshark can’t see the makeshift emulator router). Fortunately, the ADT developers provided two ways to accomplish this.

1) Using the telnet interface (start emulator first)

telnet localhost 5554
network capture start emulator.cap
-- Do Something Cool --
network capture stop

2) Using the emulator command line option

emulator -avd myavd -verbose -tcpdump emulator.cap

Afterwords simply open the cap file with Wireshark. That’s it.

Speeding up AOSP Builds

The Android Development Tools (ADT) is a massive project and is very well done. Getting your hands on and compiling the Android Open Source Project (AOSP) is very easy too. So easy you’ll quickly want to improve your compile performance. An initial compile of AOSP can take about ~46 minutes on an Intel i7 @ 3.4 GHz. Using the compile cache will definitely speed up sequential builds but more can still be done. Sacrificing just a few MB of RAM (~60 MB) for a system temp directory (/tmp) ramdisk can reduce compile time anywhere from ~2%-10% depending on total system throughput (other hardware specs). The C/C++ compiler creates, writes and deletes a temporary file for each source file crunched. Keeping this I/O traffic in RAM offers efficiency gains.

compiling

System:
PC Desktop i7 @ 3.4 GHz x 8 w/16 GB RAM and 7200 RPM Drive
Without Ramdisk: ~44 minutes
With Ramdisk: ~41 minutes
Reduction in time playing swords: ~7%

System:
PC Laptop Intel Core 2 Duo CPU T8300 @ 2.4 Gz x 2 w/4 GB RAM and 5600 RPM Drive
Without Ramdisk: 705 minutes
With Ramdisk: 650 minutes
Reduction in time playing swords: ~8%

To setup your /tmp directory to be a ramdisk on Ubuntu systems add the following to your /etc/fstab file and reboot

ramdisk /tmp tmpfs mode=1777,size=2g

There are two quick ways to make sure the ramdisk is working.
1) run the ‘df’ command and confirm /tmp is mounted with the correct size
2) copy a large file to your home directory and again to the /tmp directory and compare speeds

chris@chis-devpc:~/ServerBackups$ time cp 2012.11.04.04.00.MySQL_Backup.sql.gz ~/

real	0m2.657s
user	0m0.000s
sys	0m0.240s
chris@chis-devpc:~/ServerBackups$ time cp 2012.11.04.04.00.MySQL_Backup.sql.gz /tmp

real	0m0.224s
user	0m0.000s
sys	0m0.140s

The file copied above is 208MB. As you can see the ramdisk was an order of magnitude faster.

Git – Command Prompt Extension

If you use Git from a command line interface you probably find yourself typing ‘git status’ and ‘git branch -a’ way too many times a day. The public Git repository has a great set of tools that can be enabled post installation. I’ve found the most useful to be the Bash shell extensions used for displaying the current branch and repository status directly in the PS1 prompt. These three environment variables save me hundreds of typed characters a day:

GIT_PS1_SHOWDIRTYSTATE=1
GIT_PS1_SHOWUPSTREAM=”auto”
GIT_PS1_SHOWUNTRACKEDFILES=1

With these set in your .bash_profile script anytime you change into a Git repository directory additional information about the state of that repository is automatically appended to the prompt. If simply setting these variables doesn’t work copy the /git/contrib/completion/git-completion.bash file to your home directory and source it in your .bash_profile.

After entering a Git repository directory what is the command prompt telling me?

The branch name should be obvious. If you see a ‘*’ character it means you have unstaged changed in the repo (not committed). If you see a ‘+’ character your changes have been staged but not pushed to a remote branch. If you see a ‘%’ character it means you have created new file(s) but haven’t started tracking them yet (haven’t run ‘git add filename’). If you see a ‘>’ character it means what is checked out is ahead of the remote server. A ‘<‘ and ‘=’ character mean your checked out repository is behind or equal to the remote server.

WiX Custom Action Sequencing

WiX is an open source project sponsored by Microsoft that exposes its operating system installer functionality via XML elements. The nuances of this declarative technology and inconsistent syntax have given birth to an entire classification of engineers in the software industry called deployment engineers.  These engineers are frequently tasked with packaging and distributing executable binaries for various versions of the Windows platform. All things considered WiX has matured nicely and is becoming quite powerful. That said it can be a real pain to start using if you’re a novice due to the lack of syntax consistency and plethora of contradictory information on the topic. Not to mention ensuring platform compatibility is nearly impossible given that most people don’t have a copy of every version of Windows ever released… and yes, the installer functionality changed with every version too :-) That said, this post is intended to add to the developer WiX ‘entropy’.

Every book I’ve skimmed through and blog post I’ve read on WiX custom action sequencing is right in ‘theory’ but not always correct in practice… Through trial and error efforts along with the help of my Chief Architect we determined that WiX custom action sequencing is fairly arbitrary and to ensure consistent functionality the action should be schedule after what appears to be an undocumented sequence.

What books and other blogs say to do:

<CustomAction Id="RegisterSomething"
              Directory="INSTALLDIR"
              ExeCommand='regsvr32.exe /s "[INSTALLDIR]something.dll"'
              Return="check">
</CustomAction>

<InstallExecuteSequence>
            <Custom Action="RegisterSomething"
                    After="InstallFinalize">NOT Installed</Custom>
</InstallExecuteSequence>

What you should consider doing instead is:

<InstallExecuteSequence>
            <Custom Action="RegisterSomething"
                    After="RemoveExistingProducts">NOT Installed</Custom>
</InstallExecuteSequence>

Why? Most operations deployment engineers will want to perform during install time will require administrator (“elevated”) privileges. Which means if the Windows user is using UAC it will only run elevated if executed prior to the ‘InstallFinalize’ sequence, not after. Simply changing the original line above from ‘After’ to ‘Before’ will not work consistently either. This is because the sequencing is arbitrary. Meaning that it’s possible for your custom actions to be schedule to run before the ‘InstallExecute’ sequence… because it’s technically ‘Before’ ‘InstallFinalize’ :-) In this case you’ll be running custom actions on binaries and runtimes that haven’t been committed to the system yet.

Python JSON Client & Server Redux

The jsocket package is for use during the development of distributed systems. There are two ways to use the package. The first and simplest is to create a custom single threaded server by overloading the the jsocket.ThreadedServer class. The second, is to use the server factory functionality by overloading the jsocket.ServerFactoryThread class and passing this declaration to the jsocket.ServerFactory(FactoryThread) object. This creates a multithreaded custom JSON server for any number of simultaneous clients.

In case you care, the history of this snippet can be found in the original Python JSON Client & Server post. All of the code can be forked on GitHub here, and the latest release can be installed off PyPi using either “pip install jsocket” or “easy_install jsocket” depending on which python package manager you use.

Using some UML notation you can decipher the inheritance of the jsocket package to be six new module classes and two existing Python classes:

Regardless of which way you choose to use the package, class inheritance is mandatory. As stated above, the simplest way to leverage functionality is to create a custom single threaded server by overloading the ThreadedServer class (just like the ServerFactory class). This is a great way to prototype simple applications in which you want to transfer data in JSON format between a client.

     import jsocket
     import logging
     class MyServer(jsocket.ThreadedServer):
          # This is a basic example of a custom ThreadedServer.
          def __init__(self):
               super(MyServer, self).__init__()
               self.timeout = 2.0

          def _process_message(self, obj):
               # pure virtual method from base class
               if obj != '':
                    if obj['message'] == "new connection":
                         logging.info("new connection.")

     if __name__ == "__main__":
          client = jsocket.JsonClient()
          client.connect()
          client.send_obj({"message": "new connection"})

Albeit brief, the above example creates a server that is capable of acting on JSON data of type “{‘message': ‘new connection’}”. There are no restrictions on what you declare in the _process_message virtual method. You can think of this as defining your ‘protocol’ for the server.

The second approach is to use the ServerFactory class to create a multithreaded custom JSON server for any number of simultaneous clients. To do this you must inherit from the ServerFactoryThread class. Just like above you need to implement the virtual _process_message method to support your ‘protocol’ (that is, what kind of data you want the server to respond to). After this is done simply pass the class declaration to the ServerFactory constructor and it does the rest. Whenever the server sees a new client connection it will fork a separate thread of control to manage the interaction with that client (the thread is automatically terminated when the client disconnects).

     import jsocket
     import logging
     class MyFactoryThread(jsocket.ServerFactoryThread):
          # This is an example factory thread, which the server factory will
          # instantiate for each new connection.
          def __init__(self):
               super(MyFactoryThread, self).__init__()
               self.timeout = 2.0

          def _process_message(self, obj):
               # virtual method - Implementer must define protocol
               if obj != '':
                    if obj['message'] == "new connection":
                         logging.info("new connection.")
                    else:
                         logging.info(obj)

     if __name__ == "__main__":
          server = jsocket.ServerFactory(MyFactoryThread)
          server.timeout = 2.0
          server.start()

          #create and connect as many clients as you like here
          client = jsocket.JsonClient()
          client.connect()
          client.send_obj({"message": "new connection"})

          client.close()
          server.stop()
          server.join()

Windows Resource – Access Denied

In the Windows PC industry it is very common for applications to attached resources to an executable. Resources can be anything from bitmaps, UI components, string translations and copyright information. These resources are then used by the executable at runtime and also by Windows Explorer (for icons and properties information).

Resources can be modified post compilation by using three methods provided by MSDN: BeginUpdateResource, UpdateResource and EndUpdateResource. I recently had to use these methods and painfully discovered a nasty race condition between my resource updater application and Windows Explorer. I found information on these random failures to be scarce, hence the blog post…

When you are modifying the resource of an executable or DLL and Windows Explorer is open and showing the directory of the file being updated (regardless of visibility), a file access race condition exists that produces one of the following two errors:

The system cannot open the device or file specified.

Access denied.

The numbers I collected show an average file access failure rate of 2.3% when updating a resource that Explorer is also showing. If you are experiencing these errors simply close Windows Explorer and your resource updater application will function as expected every time. Given that Windows is closed source I can’t confirm exactly what Explorer is doing but my hypotheses is that Explorer temporarily opens and reads the resources of each file in the directory (most likely so it can determine which icon to show) and refreshes that resource when it notices a change to the file. If your updater application requests a file handle during this time you will see one of the above errors.