Tag: Android

Supporting 64-bit devices

When I left off last week, I was told that I needed to upload my app in app bundle format, instead of APK format. The documentation there may seem intimidating, but clicking through the links eventually brought me to instructions for building an app bundle with Gradle. (There are other ways to do it, but I’m already using Gradle.) It’s as simple as swapping the command I give to Gradle – instead of assembleRelease, I run bundleRelease. And it seems to work. At least, Google Play accepts the bundle.

But then Google gives me another error. I’ve created a 32-bit app, and from now on Google requires 64-bit support. I do like 64-bit code in theory, but at this stage it’s also kind of scary. I’ll need to mess with the C++ compile process, which I’m not familiar with. And I’m stuck with old versions of Lime and hxcpp, so even if they’ve added 64-bit support, I can’t use that.

Initially, I got a bit of false hope, as the documentation says “Enabling builds for your native code is as simple as adding the arm64-v8a and/or x86_64, depending on the architecture(s) you wish to support, to the ndk.abiFilters setting in your app’s ‘build.gradle’ file.” I did that, and it seemed to work. It compiled and uploaded, at least, but it turned out the app wouldn’t start because it couldn’t find libstd.so.

I knew I’d seen that error ages ago, but wasn’t sure where. Eventually after a lot of trial and error, I tracked it down to the ndk.abiFilters setting. Yep, that was it. Attempting to support 64-bit devices just breaks it for everybody, and the reason is that I don’t actually have any 64-bit shared libraries (a.k.a. .so files). This means I need to:

  1. Track down all the shared libraries the app uses.
  2. Compile 64-bit versions of each one.
  3. Include them in the build.

And I have only about a week to do it.

Tracking down shared libraries

The 32-bit app ends up using seven shared libraries: libadcolony.so, libApplicationMain.so, libjs.so, liblime-legacy.so, libregexp.so, libstd.so, and libzlib.so.

Three of them (regexp, std, and zlib) are from hxcpp, located in hxcpp’s bin/Android folder. lime-legacy is from Lime, naturally, and is found in Lime’s lime-private/ndll/Android folder. ApplicationMain is my own code, and is found inside my project’s bin/android/obj folder. From each of these locations, the shared libraries are copied into my Android project, specifically into app/src/main/jniLibs/armeabi.

The “adcolony” and “js” libraries are slightly different. Both of those are downloaded during when Gradle compiles the Android app. Obviously the former is from AdColony, and I think the latter is too. Since both have 64-bit versions, I don’t think I need to worry about them.

Interestingly, Lime already has a few different versions of liblime-legacy.so, but no 64-bit version. If I had a 64-bit version, it would go in app/src/main/jniLibs/arm64-v8a, where Gradle will look for it. (Though since I’m using an older Gradle plugin than 4.0, I may have to figure out how to include CMake IMPORTED targets, whatever that means.)

As far as I can tell, that’s a complete list. C++ is the main problem when dealing with 32- and 64-bit support. On Android, C++ code has to go in a shared object. All shared objects go inside the lib folder in an APK, and the above is a complete list of the contents of my app’s lib folder. So that’s everything. I hope.

Compiling hxcpp’s libraries

How you compile a shared library varies depending on where it’s from, so I’ll take them one at a time.

I looked at hxcpp first, and found specific instructions. Run neko build.n android from a certain folder to recompile the shared libraries. This created several versions of libstd.so, libregexp.so, and libzlib.so, including 64-bit versions. Almost no effort required.

Next, I got started working on liblime-legacy.so, but it took a while. Eventually, I realized I needed to test a simple hypothesis before I wasted too much time. Let’s review the facts:

  • When I compile for 32-bit devices only, everything works. Among other things, libstd.so is found.
  • When I compile for 32-bit and 64-bit devices, it breaks. libstd.so is not found.
  • Even though it can’t be found, libstd.so is present in the APK, inside lib/armeabi. (That’s the folder with code for 32-bit devices.)
  • lib/arm64-v8a (the one for 64-bit devices) contains only libjs.so and libadcolony.so.

My hypothesis: because the arm64-v8a folder exists, my device looks only in there and ignores armeabi. If I put libstd.so there, the app should find it. If not, then I’m not going to be able to use liblime-legacy.so either.

Test #1: The 64-bit version of libstd.so is libstd-64.so. (Unsurprisingly.) Let’s add it to the app under that name. I don’t think this will work, but I can at least make sure it ends up in the APK. Result: libstd-64.so made it into the APK, and then the app crashed because it couldn’t find libstd.so.

Test #2: Actually name it the correct thing. This is the moment of truth: when the app crashes (because it will crash), what will the error message be? Result: libstd.so made it into the APK, and then the app crashed because it couldn’t find libregexp.so. Success! That means it found the library I added.

Test #3: Add libregexp.so and libzlib.so. This test isn’t so important, but I have the files sitting around, so may as well see what happens. My guess is, liblime-legacy.so is next. Result: could not find liblime-legacy.so, as I guessed.

(For the record, I’m not doing any of this the “right” way, which means if I clear my build folder or switch to another machine, it’ll stop working. But I’ll get to that later.)

Compiling Lime’s library

Like hxcpp, Lime comes with instructions, but unlike hxcpp, they didn’t work first try. From the documentation you’d think lime rebuild android -64 would do it, but that’s for Intel processors (Android typically uses Arm). So the correct command is lime rebuild android -arm64, but even that doesn’t work.

Turns out, AndroidPlatform only compiles for three specific 32-bit architectures, and ignores any others you request. I’m going to need to add a 64-bit option there.

Let’s jump forwards in time and see what the latest version of AndroidPlatform looks like. …What do you know, it now supports 64-bit architectures. Better yet, the rest of the code is practically unchanged (they renamed a class, but that’s about it). Since it’s so similar, I should be able to copy over the new code, adjusting only the class name. Let’s give that a try…

…and I figured out why the 64-bit option wasn’t included yet. The compiler immediately crashes with a message that it can’t find stdint.h. Oh, and the error occurred inside the stdint.h file. So it went looking for stdint.h, and found it, but then stdint.h told it to find stdint.h, and it couldn’t find stdint.h. Makes sense, right?

According to the tech support cheat sheet, what you do is search the web for a few words related to the problem, then follow any advice. When I did, I found someone who had the same bug (including the error message pointing to stdint.h), and the accepted solution was to target Android 21 because that’s the first one that supports 64-bit. Following that advice, I did a find and replace, searching all of Lime’s files for “android-9” and replacing with “android-21”. And it worked.

As expected, fixing one problem just exposed another. I got an error about how casting from a pointer to int loses precision. I’m certain this is only the first of many, many similar errors, since all this code was designed around 32 bit pointers. It should be fixable, in one of several ways. As an example of a bad way to fix it, I tried changing int to long. A long can hold a 64-bit pointer, but it’s overkill on 32-bit devices, and it’s even possible that the mismatch would cause subtle errors.

But hey, with that change, the compile process succeeded. Much to my surprise. I was expecting an endless stream of errors from all the different parts of the code that aren’t 64-bit-compatible, but apparently those all turned into warnings, so I got them all at once instead of one at a time. These warnings ended up falling into three distinct groups.

  • Three warnings came from Lime-specific code. After consulting the modern version of this code, I made some educated guesses about how to proceed. First, cast values to intptr_t instead of int, because the former will automatically adjust for 64 bits. (Actually I went with uintptr_t, but it probably doesn’t matter.) Second, when the pointer is passed to Haxe code, pass it as a Float value, because in Haxe that’s a 64-bit value. Third, acknowledge that step 2 was very weird and proceed anyway, hoping it doesn’t matter.
  • A large number of warnings came from OpenAL (an open-source audio library, much like how OpenGL is an open-source graphics library and OpenFL is an open-source Flash library). I was worried that I’d have to fix them all by hand, but eventually I stumbled across a variable to toggle 32- vs. 64-bit compilation. But luckily, the library already supported 64 bits, and I just had to enable it. (Much safer than letting me implement it.)
  • cURL produced one warning – apparently it truncates a pointer to use as a random seed. I don’t know if that’s a good idea, but I do know the warning is irrelevant. srand works equally well if you give it a full 32-bit pointer or half of a 64-bit pointer.

Ignoring the cURL warning, the build proceeded smoothly. Four down, one to go.

Copying files the correct way

As I mentioned earlier, I copied hxcpp’s libraries by hand, which is a temporary measure. The correct way to copy them into the app is through Lime, specifically AndroidPlatform.hx. Like last time I mentioned that file, this version only supports 32-bit architectures, but the latest version supports more. Like before, my plan is copy the new version of the function, and make a few updates so it matches the old.

Then hit compile, and if all goes well, it should copy over the 64-bit version of the four shared libraries I’ve spent all week creating. And if I’m extra lucky, they’ll even make it into the APK. Fingers crossed, compiling now…

Compilation done. Let’s see the results. In the Android project, we look under jniLibs/arm64-v8a, and find:

  1. libApplicationMain.so
  2. liblime-legacy.so
  3. libregexp.so
  4. libstd.so
  5. libzlib.so

Hey, cool, five out of four libraries were copied successfully. (Surprise!)

I might’ve glossed over this at the start of this section, but AndroidPlatform.hx is what compiles libApplicationMain.so. When I enabled the arm64 architecture to make it copy all the libraries, it also compiled my Haxe code for 64 bits and copied that in. On the first try, too.

Hey look at that, I have what should be a complete APK. Time to install it. Results: it works! The main menu shows up (itself a huge step), and not only that, I successfully started the game and played for a few seconds.

More audio problems

And then it crashed when I turned the music on, because apparently OpenAL still doesn’t work. There was a stack trace showing where the null pointer error happened, but I had to dig around some more to figure out where the variable was supposed to be set. (It was actually in a totally different class, and that class had printed an error message but I’d ignored the error message because it happened well before the crash.)

Anyway, the problem was it couldn’t open libOpenSLES.so, even though that library does exist on the device. And dlerror() returned nothing, so I was stumped for a while. I wrote a quick if statement to keep it from crashing, and resigned myself to a silent game for the time being.

After sleeping on it, I poked around a little more. Tried several things without making progress, but then I had the idea to try loading the library in Java. Maybe Java could give me a better error message. And it did! “dlopen failed: "/system/lib/libOpenSLES.so" is 32-bit instead of 64-bit.” Wait, why does my 64-bit phone not have 64-bit libraries? Let me take another look… aha, there’s also a /system/lib64 folder containing another copy of libOpenSLES.so. I bet that’s the one I want. Results: it now loads the library, but it freezes when it tries to play sound.

It didn’t take too long to track the freeze down to a threading issue. It took a little longer to figure out why it refused to create this particular thread. It works fine in the 32-bit version, but apparently “round robin” scheduling mode is disallowed in 64-bit code. Worse, there’s very little information to be found online, and what little there is seems to be for desktop machines with Intel processors, not Android devices with Arm processors. My solution: use the default scheduling mode/priority instead of round robin + max priority. This seems to work on my device, and the quality seems unaffected. Hopefully it holds up on lower-end devices too.

Conclusion

And that’s about it for this post. It was a lot of work, and I’m very glad it came together in the end.

Looking back… very little of this code will be useful in the future. These are stopgap measures until Runaway catches up. Once that happens, I can use the latest versions of Lime and OpenFL, which support 64-bit apps (and do a better job of it than I did here). I will be happy once I can consign this code to the archives and never deal with it again.

But.

Work like this is never wasted. The code may not be widely useful, but I’ve learned a lot about Android, C++, and how the two interact. I’ve learned about Lime, too, digging deeper into its build process, its native code, and several of its submodules. (Which will definitely come in handy because I’m one of the ones responsible for its future.)

The app is just about ready for release, with about a week to spare. But I still have a couple things to tidy up, and Kongregate would like some more time to test, so we’re going to aim for the middle of next week, still giving us a few days to spare.

Testing the Google Play Billing Library

To recap, I’ve been working to update the Google Play Billing Library, because if I don’t update it by November 1, Google will lock the app and I’ll never be able to update it again. Pretty important.

When I left off last week, I’d rewritten a lot of code, but hadn’t been able to test it due to an infinite loop. This week, I was hoping to fix the infinite loop quickly and move on to the many bugs in my as-yet-untested billing library integration.

Tracking down the infinite loop

This was a tough bug. While it’s far from the toughest I’ve ever tracked down, it seemed mysterious enough that shortly before finding it, I was nearly ready to give up and look for a workaround. (There’s a motivational poster in there somewhere.) But I’m getting ahead of myself.

At first, I had almost nothing to go off of. The app would start, log a few unrelated messages, and then hang with a blank screen, with no clue as to why. The lag (and eventual “app not responding” message) tipped me off that it was an infinite loop, but that isn’t much to go on.

My biggest clue (or so I thought) was knowing what code most recently changed: the billing library. So somehow that must be looping forever, right? Well… I commented out the “connect to billing library” code, and nothing changed. I commented out more and more features of the library, followed by lots of other related and barely-related code, all to no avail.

I finally got the app to run by disabling all internet-related libraries and features (I have a compiler flag for that), but nothing short of that worked. I started to suspect that the mere presence of the billing library caused this error, even without using it in any way. I considered dropping all network features and releasing the app with no logins, purchases, or cloud saves, since that was starting to seem easier than tracking this down. Desperate times and all that.

Then I remembered I wasn’t out of debugging techniques just yet. Next up was version control. I could rewind the clock and get a version that worked, and carefully examine the changes from there. So I reverted the GPBL and my last few weeks of code, and compiled the app the way I’d had it working.

And it still froze. Huh.

This is a lesson I’ve learned many times over, but still sometimes forget. When something makes zero sense, you need to check your assumptions. I’d assumed that since this was a new error, my new code was at fault. But now my old code didn’t work either, and that meant I was looking in the wrong place.

I forget exactly what it was I did, but knowing this, I managed to make the app crash. That’s a good thing: with a crash comes an error message, and (at least in this case) a stack trace. This instantly narrowed the search down to a single line of code, and the line of code had nothing to do with billing.

Teaching my code patience

Turns out, I’ve been overlooking a significant issue for a while now. To make a medium-length story short, cloud save setup happens partway through the game’s setup process. I don’t know (or care) exactly when, because it shouldn’t matter.

At this point in the process, it sends a request to log in to PlayFab. You have to log in eventually if you want to use cloud saves, so I figured, might as well start it immediately so it can happen in the background. What I forgot is that Haxe’s network requests (usually) don’t happen in the background. Instead, the login command puts the rest of the setup process on hold until it connects.

Not only that, the moment you finish connecting, several different classes try to make use of the cloud data. But because game setup wasn’t done, some important variables were still null, including PlayFab.instance. When PlayFab.instance is null, it runs the PlayFab class constructor. The constructor sends a request to log in. After logging in, classes try to make use of the cloud data. But PlayFab.instance is still null… so it runs the constructor again. And again.

There’s a long-running programmer joke that you start out wondering why your code doesn’t work, and end up wondering how it ever worked. That’s me right now. (And I particularly relate to the second comment on that post: “Or if you change your code to make it work better, it doesn’t work, so you load your backup and your original code doesn’t work anymore…”)

The solution? Just don’t log in until the game is fully ready. A little bit of patience goes a long way. (I’ve also pushed a commit in case the infinite loop comes up elsewhere.)

Surprisingly, everything above only took a couple days. It felt like longer, but my computer’s clock disagrees.

Testing the GPBL (finally)

So my plan was, after resolving the infinite loop, I’d spend the rest of the week testing the billing code. This would provide lots more material for the epic conclusion to my “why developing for Android is so annoying” series.

But this plan failed, because I only ran into two problems. The first came with an error message, and the error message told me exactly how to fix it. The second required a bit of effort to figure out why things weren’t happening in order, but it wasn’t bad enough to blog about. And then… purchases just worked.

I spent nearly half the week just kind of waiting to see if Google Play would accept or reject the app. (Answer: reject. Google now insists on app bundles, so that’s what I’m working on next.)

So, uh… maybe mobile development isn’t quite as bad as I remember.

Updating the Google Play Billing Library

As I’ve mentioned a couple posts ago, I need to update this specific library by November. Google was nice enough to provide a migration guide to help with the process, though due to my own setup, it isn’t quite as easy as that.

As I start writing this post, I’ve just spent several hours restructuring Java code, and I anticipate several more. I’ve been tabbing back and forth between the guide linked above and the API documentation, going back and forth as to class structure. I’m trying to track in-app purchases (you know, the ones bought with actual money), but this is made harder by the fact that Google doesn’t, you know, track those purchases.

The problem with purchases on Google Play

Let me introduce you to the concept of “consuming” purchases. On the Play Store, each in-app purchase has two states: “owned” and “not owned.” If owned, you can’t buy it again. This is a problem for repeatable purchases (like in Run Mobile), because you’re supposed to be able to buy each option as much as you like. Google’s solution is this: games should record the purchase, then “consume” it, setting it back to “not owned” and allowing the player to buy it again.

I know for a fact Google keeps track of how many times each player has purchased each item, but that information is not available to the game. I get a distressingly high number of reports that people lost their data; would’ve been nice to have a backup, but nope.

There’s one upside. In the new version of the library, Google allows you to look up the last time each item was bought. So that’s a partial backup; even if all the other data was deleted, the game can award that one purchase. (Actually up to four purchases, one per price point.) That’ll be the plan, anyway.

The problem with templates

Templates are a neat little feature offered by Haxe and enhanced by Lime. As I’ve mentioned before, Lime doesn’t create Android apps directly; it actually creates an Android project and allows the official Android SDK to build the app. Normally, Lime has its own set of templates to build the project from, but you can set your own if you prefer.

That’s how I write Java code. I write Java files and then tell Lime “these are templates; copy them into the project.” Now, Lime doesn’t do a direct copy; it actually does some processing first. This processing step means I get to adjust settings. Sometimes I mark code as being only for debug builds, or only for the Amazon version of the app. (Yeah, it’s on Amazon.)

Essentially, this functions as conditional compilation, which is a feature I use extensively in Haxe code. When I saw the opportunity to do the same thing in Java, I jumped on it.

Problem is, the template files are not (quite) valid Java code. This makes most code helpers nigh-unusable. Well, as far as I know. Since I never expected any to work, I didn’t try very hard to make it happen. Instead, I just coded without any quality of life features (code completion, instant error checking, etc.). Guess what happens when you don’t have quality of life features? Yep, your life quality suffers.

Use the best tools for each part: Xcode to make iOS code, Android Studio to make Android code and VSCode (or your other favourite Haxe editor) to make Haxe code

—Some quality advice that I never followed.

You know, I’ve always hated working on the “mobile” parts of Run Mobile. I’d do it, but only reluctantly, and it’d always be slow going, no matter how simple and straightforward everything seemed on paper. When I was done and could get back to Haxe, it’d feel like a weight off my chest. In retrospect, I think the lack of code completion was a big part of it. (The other part being that billing is outside my comfort zone.)

I’m not going to change up my development process just yet. There’s too much riding on me getting this done, and not nearly enough time to change my development process. But eventually, I’m going to see about writing the Java code in Android Studio.

Conclusion

After even more hours of work, I’ve started to get the hang of the new library, rewritten hundreds of lines of code, and fixed a few dozen compile errors. I’ve removed a few features along the way, but hopefully nothing that impacts end users.

In retrospect, the conversion guide wasn’t very helpful. It provided “before” and “after” code examples, but the “before” code looked nothing like my code, so I couldn’t be sure what to replace with what. The API docs were far more useful – since everything centers on BillingClient, I could always start reading there.

As of right now, the app compiles, but that’s it. When launched, it immediately hangs, probably because of an infinite loop somewhere. Once that’s fixed, it’s on to the testing phase.

Coding in multiple languages

Quick: what programming language is Run 3 written in? Haxe, of course. (It’s even in the title of my blog.) But that isn’t all. Check out this list:

  • Haxe
  • Neko
  • Java
  • JavaScript
  • ActionScript
  • C++
  • Objective-C
  • Python
  • Batch
  • Bash
  • Groovy

Those are the programming languages that are (or were) involved in the development of Run 3/Run Mobile. Some are on the list because of Haxe’s capabilities, others because of Haxe’s limits.

Haxe’s defining feature is its ability to compile to other languages. This is great if you want to write a game for multiple platforms. JavaScript runs in browsers, C++ runs on desktop, Neko compiles quickly during testing, ActionScript… well, we don’t talk about ActionScript anymore. And that’s why those four are on the list.

Batch and Bash are good at performing simple file operations. Copying files, cleaning up old folders, etc. That’s also why Python is on the list: I have a Python file that runs after each build and performs simple file operations. Add 1 to the build count, create a zip file from a certain folder, etc. Honestly it doesn’t make much difference which language you use for the simple stuff, and I don’t remember why I chose Python. Nowadays I’d definitely choose Haxe for consistency.

The rest are due to mobile apps. Android apps are written in Java or Kotlin, then compiled using Groovy. Haxe does compile to Java, but it has a reputation for being slow. Therefore, OpenFL tries to use as much C++ as possible, only briefly using Java to get the game up and running.

iOS is similar: apps are typically written in Objective-C or Swift, and Haxe doesn’t compile to either of those. But you can have a simple Objective-C file start the app, then switch to C++.

Even leaving aside Python, Batch, and Bash, that’s a lot of languages. Some of them are independent, but others have to run at the same time and even interact. How does all that work?

Source-to-source compilation

Let’s start with source-to-source compilation (the thing I said was Haxe’s defining feature) and what it means. Suppose I’m programming in Haxe and compiling to JavaScript.

Now, by default Haxe code can only call other Haxe code. Say there’s a fancy JavaScript library that does calligraphy, and I want to draw some large shiny letters. If I was writing JavaScript, I could call the drawCalligraphy() function no problem, but not in Haxe.

To accomplish the same thing in Haxe, I need some special syntax to insert JavaScript code. Something like this:

//Haxe code (what I write)
var fruitOptions = ["apple", "orange", "pear", "lemon"];
var randomFruit = fruitOptions[Std.int(Math.random() * fruitOptions.length)];
js.Syntax.code('drawCalligraphy(randomFruit);');
someHaxeFunction(randomFruit);

//JS code (generated when the above is compiled)
let fruitOptions = ["apple","orange","pear","lemon"];
let randomFruit = fruitOptions[Math.random() * fruitOptions.length | 0];
drawCalligraphy(randomFruit);
someHaxeFunction(randomFruit);

Note how similar the Haxe and JavaScript functions end up being. It almost feels like I shouldn’t need the special syntax at all. As you can see from the final line of code, function calls are left unchanged. If I typed drawCalligraphy(randomFruit) in Haxe, it would become drawCalligraphy(randomFruit) in JavaScript, which would work perfectly. Problem is, it doesn’t compile. `drawCalligraphy` isn’t a Haxe function, so Haxe throws an error.

Well, that’s where externs come in. By declaring an “extern” function, I tell Haxe “this function will exist at runtime, so don’t throw compile errors when you see it.” (As a side-effect, I’d better type the function name right, because Haxe won’t check my work.)

tl;dr: Since Haxe creates code in another programming language, you can talk to other code in that language. If you compile to JS, you can talk to JS.

Starting an iOS app

Each Android or iOS app has a single defined “entry point,” which has to be a Java/Kotlin/Objective-C/Swift file. Haxe can compile to (some of) these, but there’s really no point. It’s easier and better to literally type out a Java/Kotlin/Objective-C/Swift file, which is exactly what Lime does.

I’ve written about this before, but as a refresher, Lime creates an Xcode project as one step along the way to making an iOS app. At this point, the Haxe code has already been compiled into C++, in a form usable on iOS. Lime then copy-pastes in some Objective-C files and an Xcode project file, which Xcode compiles to make a fully-working iOS app. (And it’s a real project; you could even edit it in Xcode, though that isn’t recommended.)

And that’s enough to get the app going. When compiled side-by-side, C++ and Objective-C++ can talk to one another, as easily as JavaScript can communicate with JavaScript. Main.mm (the Objective-C entry point) calls a C++ function, which calls another C++ function, and so on until eventually one of them calls the compiled Haxe function. Not as simple as it could be, but it has the potential to be quite straightforward.

Unlike Android.

Shared libraries

A shared library or shared object is a file that is intended to be shared by executable files and further shared object files. Modules used by a program are loaded from individual shared objects into memory at load time or runtime, rather than being copied by a linker when it creates a single monolithic executable file for the program.

Traditionally, shared library/object files are toolkits. Each handles a single task (or group of related tasks), like network connections or 3D graphics. The “shared” part of the name means many different programs can use the library at once, which is great if you have a dozen programs connecting to the net and don’t want to have to download a dozen copies of the network connection library.

I mention this to highlight that Lime does something odd when compiling for Android. All of your Haxe-turned-C++ code goes in one big shared object file named libApplicationMain.so. But this “shared” object never gets shared. It’s only ever used by one app, because, well, it is that app. Everything outside of libApplicationMain.so is essentially window dressing; it’s there to get the C++ code started. I’m not saying Lime is wrong to do this (in fact, the NDK documentation tells you to do it), I’m just commenting on the linguistic drift.

To get the app started, Lime loads the shared object and then passes the name of the main C++ function to SDL, which loads the function and then calls it. Bit roundabout, but whatever works.

tl;dr: A shared library is a pre-compiled group of code. Before calling a function, you need two steps: load the library, then load the function. On Android, one of these functions is basically “run the entire app.”

Accessing Android/iOS features from Haxe

If your Haxe code is going into a shared object, then tools like externs won’t work. How does a shared object send messages to Java/Objective-C? I’ve actually answered this one before with examples, but I didn’t really explain why, so I’ll try to do that.

  • On Android, you call JNI.createStaticMethod() to get a reference to a single Java function, as long as the Java function is declared publicly. Once you have this reference, you can call the Java function. If you want more functions, you call JNI (Java Native Interface) multiple times.
  • On iOS, you call CFFI.load() to get a reference to a single C (Objective or otherwise) function, as long as the C function is a public member of a shared library. Once you have this reference, you can call the C function. If you want more functions, you call CFFI (C Foreign Function Interface) multiple times.

Gotta say, there are a lot of similarities, and I’m guessing that isn’t a coincidence. Lime is actually doing a lot of work under the hood in both cases, with the end goal of keeping them simple.

But wait a minute. Why is iOS using shared libraries all of a sudden? We’re compiling to C++ and talking to Objective-C; shouldn’t extern functions be enough? In fact, they are enough. Shared libraries are optional here, though recommended for organization and code consistency.

You might also note that last time I described calling a shared library, it took extra steps (load the library, load the function, call the function). This is some of the work Lime does under the hood. The CFFI class combines the “load library” and “load function” steps into one, keeping any open libraries for later use. (Whereas C++ doesn’t really do “convenience.”)

tl;dr: On Android, Haxe code can call Java functions using JNI. iOS extensions are designed to mimic this arrangement, though you use CFFI instead of JNI.

Why I wrote this post

Looking back after writing this, I have to admit it’s one of my less-informative blog posts. I took a deep dive into how Lime works, yes, but very little here is useful to an average OpenFL/Lime user. If you want to use CFFI or JNI, you’d be better off reading my old blog post instead.

Originally, this post was supposed to be a couple paragraphs leading into to another Android progress report. (And I’d categorized it under “development,” which is hardly accurate.) But the more I wrote, the clearer it became that I wasn’t going to get to the progress report. I almost abandoned this post, but I was learning new things, so I decided to put it out there.

(For instance, it had never occurred to me that CFFI was optional on iOS. It may well be the best option, but since it is just an option rather than mandatory, I’ll want to double-check.)

Why I haven’t updated Run Mobile in ages (part 1)

Google has announced a November 1 deadline to update Play Store apps, and I’ve been keeping an eye on that one. We’re now getting close enough to the deadline that I’m officially giving up on releasing new content with the update, and instead I’ll just release the same version with the new required library.

But why did it take this long for me to decide that? Why didn’t I do this a year ago when Google made their announcement, and keep working in the meantime? To answer that question, this blog post will document my journey to make a single small change to Run Mobile. The first step, of course, is to make sure it still compiles.

I should mention that I have two computers that I’ve used to compile Android apps. A Linux desktop, and a Windows laptop.

The Linux machine:

  • Is where I do practically all my work nowadays.
  • Performs well.
  • Is the one you’ve seen if you’ve watched my streams.
  • Has never, if I recall correctly, compiled a release copy of Run Mobile.

The Windows machine:

  • Hasn’t seen use in years.
  • Is getting old and slow; probably needs a fresh install.
  • Has the exact code I used to compile the currently-released version of Run Mobile.

Compiling on Windows

I tried this second, so let’s talk about it first. That makes sense, right?

Well, I found that I had some commits I hadn’t uploaded. Figured I’d do that real quick, and it turns out that Git is broken somehow. Not sure why, but it always rejects my SSH key. I restarted the machine, reuploaded the key to GitHub, tried both command line and TortiseGit, and even tried GitHub’s app which promises that you don’t have to bother with SSH keys. Everything still failed. At some point I’ll reinstall Git, but that’s for later. My goal here is to compile.

Fortunately, almost all of my code was untouched since the last time I compiled, and so I compiled to Neko. No dice. There were syntax errors, null pointers, and a dozen rendering crashes. Oh right, I never compiled this version for Neko, because I always targeted Flash instead.

So I stopped trying to fix Neko, and compiled for Android. And… well, there certainly were errors, but I’ll get to them later. Eventually, I fixed enough errors that it compiled. Hooray!

But for some reason I couldn’t get the machine to connect to my phone, so I couldn’t install and test the app. Tried multiple cables, multiple USB ports… nothing. And that was the last straw. This laptop was frustrating enough when Git and adb worked.

Compiling on Linux

Since Git does work on this machine, I was easily able to roll my code back. I’d even made a list of what libraries needed to be rolled back, and how far. (This is far from the first time I’ve had to do this.)

With all my code restored, I tried compiling. Result: lime/graphics/cairo/Cairo.hx:35: characters 13-21 : Unexpected operator. A single error, presumably hiding many more. Out of curiosity, I checked the file in question, expecting to see a plus sign in a weird spot, or an extra pair of square brackets around something, or who knows. Instead I found the word “operator”. (Once again I have fallen victim to the use-mention distinction.) Apparently Haxe 4.0.0 made “operator” a keyword, and Lime had to work around it.

Right, right. I’d gone back to old versions of my code, but I hadn’t downgraded Haxe. I’d assumed doing so would be difficult, possibly requiring me to download an untrustworthy file. This was the point in the process when I tried to compile on Windows instead. As explained above, that fell through, so I came back and discovered I could get it straight from the Haxe Foundation. (I’d been looking in the wrong place.) Once I reverted Haxe, that first error went away.

But that was only error #1. Fixing it revealed a new one, and fixing that revealed yet another. Rinse and repeat for an unknown number of times. Plus side, it’s simpler to keep track of a single error at a time than 20 at once. Minus side, there were a lot of errors.

  1. Type not found: haxe.Exception – Apparently I hadn’t downgraded all my libraries. After some file searching, I found two likely culprits and downgraded both.
  2. Cannot create closure with more than 5 arguments – I’ve never seen this one before, and neither has Google. I never even knew that function bindings were closures. Also, I’m not sure how addRectangle.bind(104, 0x000000, 0) has more than 5 arguments (perhaps it counts the optional arguments). But this wasn’t worth a lot of time, so I used an anonymous function to do the same thing.
  3. Invalid field access : lastIndexOf – This often comes up when a string is null. Can’t search for the last index of a letter if the string isn’t there. Fortunately I’d already run into this bug on Windows and knew the solution. Haxe 3.4 tells you to use Sys.programPath() instead of Sys.executablePath(), except `programPath` is broken.
  4. Extern constructor could not be inlined – Another error stemming from an old version of Haxe, this comes up when you cache data between compilations. It can be fixed by updating Haxe (not an option in my case) or turning off VSHaxe’s language server.
  5. Invalid field access : __s – Another null pointer error I’d already seen. But it was at this point that I remembered not to try to compile for Neko, so I turned my focus to Android instead.
  6. You need to run "lime setup android" before you can use the Android target – So of course that wasn’t going to be easy either. Apparently I’d never told Lime where to find some crucial files. (Also apparently I’d never downloaded the [NDK](https://developer.android.com/ndk/), meaning I’ve never used this machine to compile for Android.)
  7. Type not found : Jni – Wait, I (vaguely) remember that class. Why is it missing? _One search later…_ Aha, it’s still there, it’s just missing some tweaks I made on the Windows computer. This is all for some debug info that rarely matters, so I removed it for the time being.
  8. arm-linux-androideabi-g++: not found – Uh oh. This is an error in hxcpp, a library that I try very hard to avoid dealing with. Android seems to have retired the “standalone toolchains” feature this old version of hxcpp uses, and I’ve long since established that newer versions of hxcpp are incompatible. Well, I tried using `HXCPP_VERBOSE` to get more info, and while it helped with a few hidden errors, I spent way too long digging into hxcpp without making much progress. Instead, I went all the way back to NDK r18.
  9. 'typeinfo' file not found – Another C++ error, great. Seems I’m not the first OpenFL dev to run into this one, which is actually good because it lets me know which NDK version I actually need: r15c. The Android SDK manager only goes as far back as r16, so I did a manual download.
  10. gradlew: not found – It might be another “not found” error, but make no mistake, this is huge progress. All the C++ files (over 99% of the game) compiled successfully, and it had reached the Gradle/Java portion, something I’m far more familiar with. Not that I needed to be because, someone else already fixed it. The only reason I was still seeing the error is because I couldn’t use newer versions of OpenFL. One quick copy-paste later, and…
  11. Cannot convert URL '[Kongregate SDK file]' to a file. – No kidding it can’t find the Kongregate SDK; I hard-coded a file path that only exists on Windows. In retrospect I should have used a relative path, but for now I hard-coded a different path. Then, to make absolutely certain I had the right versions, I copied the Kongregate SDK (and other Java libraries) from my laptop.
  12. Could not GET 'https://dl.bintray.com/[...]'. Received status code 403 from server: Forbidden – There were nine or ten of these errors, each with a different url. Apparently Bintray has been shut down, which means everyone has found somewhere else to host their files. I looked up the new URLs and plugged them in. Surprisingly, the new urls worked first try.

And, finally, it compiled.

Closing thoughts

And that’s why I haven’t updated Run Mobile in ages. Every time I try to compile, I have to wade through a slew of bugs. Not usually this many, but there’s always something, and I’ve learned to associate updating mobile with frustration.

I was hoping to avoid this whole process. I’d hoped to finish Runaway, allowing me to use the latest versions of OpenFL, hxcpp, and the Android SDK. But there just wasn’t enough time.

Don’t get me wrong, it feels good to have accomplished this. But as a reminder, I’ve made zero improvements thus far. I haven’t copied over any new content, I haven’t updated any libraries, I haven’t even touched the Google Play Billing Library (you know, the one that must be updated by November). I’ve spent two weeks just trying to get back what I already had.

Maybe I’m being too pessimistic here. I have, in fact, made progress since February 2018. My code now compiles on Linux, unlike in 2018. My 2018 code relied on Bintray, which is now gone. And it’s possible that new content may have been included without me even trying.

And that’s enough for today. Join me next time, on my journey to make a single small change to Run Mobile.

OpenFL Template Basics: Overriding AndroidManifest.xml

What do you do if you need to change AndroidManifest.xml, and OpenFL’s customization options aren’t enough? Sure you can add permissions using <config:android permission="android.permission.WHATEVER" />, and change the app title, and so on…

But what if you want to support multiple screen sizes? You need to include a <supports-screens> tag, or Google may hide your app from tablet users! This is yet another of the billions of scenarios that OpenFL fails to account for. (Sheesh, OpenFL dev team, get a move on.)

What you need is full control of the app’s AndroidManifest.xml. You can edit Lime’s copy, but that’ll change it for every project, and it’ll get reverted when you update Lime. You can edit the copy in Export/android, but that’ll get reverted the very next time you compile. You can make an extension, but that’s overkill.

Creating Your Own Manifest File

First things first. Before you can overwrite AndroidManifest.xml, you need to define what exactly you want to overwrite it with.

Conveniently, Lime has a whole folder of template files you can copy from. In this example, we’re replacing an Android template, so we look inside templates/android. A quick search turns up the file inside templates/android/template/app/src/main. (The “app/src/main” part will be important later; remember it.)

Copy AndroidManifest.xml into your project. I usually make a “templates” folder for files like this, but if you have your own way to organize it, that’s fine too. Open up your copy of the file and add the following somewhere between <manifest> and </manifest>:

<supports-screens
    android:anydensity="true"
    android:smallscreens="true"
    android:normalscreens="true"
    android:largescreens="true"
    android:xlargescreens="true" />

Using Your Manifest File

Finally! Enough preparation, it’s time to impose our will upon Lime’s build process. No longer shall we be confined to normalScreens, we shall reach out and conquer anyDensity!

What’s that? “Get on with it”? Oh fine…

Type this in your project.xml file:

<template path="templates/AndroidManifest.xml" rename="app/src/main/AndroidManifest.xml" />

Yep, that’s it. path tells it where (in your Haxe project) to get the file from, and rename tells it where (in your Android project) to put it. In this case, it goes in app/src/main.

Which Files Can be Overridden?

To answer this question, first compile your project for Android, and open Export/android/bin (or bin/android/bin). Take a good look – everything the light touches is your kingdom. By which I mean you can override the files in that folder.

However, you need to make sure put your file in the same folder as the file you’re overriding. GameActivity.java is located in app/src/main/java/org/haxe/lime, so you’d put this in project.xml:

<template path="templates/GameActivity.java" rename="app/src/main/java/org/haxe/lime/GameActivity.java" />