iOS audio done!

It seems the most serious issues with audio sync on iOS have been fixed. By capturing video frames on ENTER_FRAME, it was necessary to insert some kind of time “gateway” as when to write a frame or not given the desired target video fps. In general, you want the library to write every other frame if you’re targeting lower fps than your game is running at (which you should). As soon as this was done, the sync issues went away. This gateway is in the native code, taking advantage of the precise measurements of time elapsed on iOS.

But majority of work was spent on rewriting the “temporary” audio recording into WAV. Now both the audio recording and encoding are happening realtime, the audio is being encoded into AAC and inserted into the MP4.

So to sum up, now the audio seems to be in sync all the time, for any video fps. For example your gameplay is 60fps, your video is 24 fps, your audio should still be in sync within the 24 fps video, even originally it was in sync at 60fps. The only exception would be if you tried to record at fps higher than the game is capable to render. That might mess things up. For example if you recorded the video at 60fps but your game rendered only at 50fps, you would be “missing” video frames and got the out of sync with audio again.

This is a situation you don’t want to get into in the first place. You might just play it safe and set the video fps to something like 20fps. Or even better, you can do a little test recording, measure the fps and set your “real recording” target fps 15%-25% lower to leave a buffer for any “bumps” in the fps (or perhaps if the target is under acceptable number such as 20, resolution of video would need to be decreased as well).

Which is a good idea for a new library feature. One method to determine the ideal fps for your videos so you’re not stuck with one fps, but instead people with more powerful iPads can record at better quality 🙂

Anyway, coming up before the next version release are still 1 or 2 fixes. Right now I’m trying to completely get rid off post processing of the video on iOS. It can take quite a long time(think ~10 seconds for 1 minute of video recording…on one of the latest iPads). It’s async, so it doesn’ t freeze the game or anything, but it’s still annoying to wait after the recording was finished. This is mostly only to flip the video upside down, because of OpenGL. In the next iOS release, flipping will very likely be done straight on the texture by OpenGL to avoid the costly post processing.