Getting slower at learning, getting better at getting distracted

My first few software jobs were largely in assembly language in an era where CPU instruction sets were pretty simple. And a lot of things were on “bare hardware”. So all you needed to do was read the manual on the processor and study the hardware schematics and you were pretty much up to speed on a new project. That has changed over the decades.

Nowadays it is all about reusable code

The most productive way for creating new programs and software systems is to reuse code. Basically, writing good code is hard and slow so if you can find code already written and debugged that you can use you are way ahead in the game.

The first software to become easily reusable was the standard libraries that each compiler comes with. No reason to write that sine or cosine routine you need to manipulate your data if there is a standard math library delivered with the compiler.

Next is a convergence of operating systems with Unix winning the war. The days when Data General, Digital Equipment, Univac, IBM, et al delivered radically different operating systems each with their own unique software libraries is over. There is still Windows but it seems most everything else is some flavor of Unix under the cover even if they look radically different to the end user. Might be based on a Berkeley System Distribution (BSD) like the Macintosh OS or on Linux like the Android OS, but it is usually Unix where the software meets the hardware. And a large bit of Unix is actually a standardized set of libraries.

Newer languages like C++ or Java now come with a much larger set of libraries than the skimpy little things that came with Fortran, Pascal or the early releases of C.

Now between relative standard Unix libraries and a hugely augmented set of language based libraries a software engineer needs to know far more than a simple CPU instruction set to be productive. There can be hundreds or even thousands of “standard libraries” you can chose between to help speed the process of making a program. If you don’t know the libraries available to you then you are at a big disadvantage because you will be writing code that your competitor isn’t so you are likely to be later to market.

Rusty, old, slow, thinking techie meets Android

Why the rambling? I‘ve been attempting to create an Android software component that might be of more use than an app that simply puts “Hello World!” on the screen.

An Android app is basically a collection of Java objects that is run by a framework in one or more Linux threads (lightweight processes). The app can pull from a huge library of standard Java packages and another huge set of Android specific packages. And, at least on the Android side, those packages are constantly changing.

Google documents everything in a form useful for reference. But not, at least for me, very useful as a tutorial. There are a huge number of tutorials and examples on the web and in print but it seems that many of them are dated and use deprecated packages and/or methods.

My poor old brain is not having as much fun as I’d hoped with this task.

Learning Java is relatively easy: It has C based syntax and semantics, object oriented blocks looking much the same as objects in other languages, etc. But the libraries are all new to me. And the Android specific libraries seem to be a shifting mass of quicksand with whole ways of doing things going from being the best thing since sliced bread to deprecated in a single OS version release.

Case in point

The program I am working on downloads a gzipped file that contains tens of millions of records. No room on the phone, or desire on my part, to actually download the file, then unzip it, then scan it for desired records. So the app is piping the HTTP stream through a stream object/package that ungzips and then through another object/package that detects CSV records and then finally takes selected records and inserts them into a Sqlite3 database. First cut took 35ms per record which basically implied running that one app on the phone for a couple of days nonstop.

Obviously, this can’t be done in the thread that is running the user interface (UI). So a quick look around suggests starting an asynchronous task/thread (conveniently named AsyncTask) and having it do the work. So that is how I did it because I was dumb and had not really internalized the “life cycle” of UI based “Activities”.

The problem is that Android can and will kill a UI Activity for a multitude of reasons. Including a change in orientation of the phone. So start my program in portrait mode and then rotate it to landscape and the old Activity instance is dropped (but not the way examples are coded the AsyncTask) and a new Activity instance is created which creates a new AsyncTask instance trying to do the same multiple day download. Not a good thing to have multiple huge background threads chewing up your bandwidth, memory, processor and storage. And not one that dawned on me when I saw all the examples and explanations, even on Google’s Android documentation, describing how to to that.

It seems the preferred way to to this is have the AsyncTask started by a “Fragment” object and then assure that there is only one instance of that Fragment object in existence using a “Fragment Manager”. At least I think that is the preferred way. Maybe there is another way of assuring a singleton instance of an object that is preferred. Or will be preferred tomorrow.

A couple of wasted days figuring out that one when what I really wanted to do was figure out if I could speed up processing enough that the job could actually be done on the phone at all.

Turns out that by applying some relatively straight forward optimizations I have gotten it speeded up to about 0.12ms per record. Not fast, but maybe fast enough for someone to do once a month to update their database.

But I still don’t have all the bases covered in the interaction between my UI Activity, the singleton UI fragment and the asynchronous background task which is making the whole app pretty flaky and definitely “pre-Alpha” quality.

Old brains, at least my old brain, is not as quick as I’d like at internalizing how various processes/threads interact with the various Java objects and the moving target that is Android’s library of object packages.