iOS: Upcoming ReplayKit support & custom resolutions

Even the main focus is Android these days, we’re not forgetting iOS either. In iOS 9, a new video recording API was introduced, called ReplayKit. We are preparing support for this API. The main advantage, besides the ease of use when implementing the native code (which is not a concern for AIR devs anyway), is full audio recording support (at least if the videos we saw don’t lie). It should be possible to include microphone as well, but otherwise what the game is playing out of speakers appears to be recorded in the video.

ReplayKit is completely separate from the FW video capture path which uses AVFoundation & OpenGL ES, so there are some disadvantages as well. You won’t be able to configure the video quality or resolution (this is also an upcoming feature of the next FW version, see below).

The biggest downside however appers to be the inability to access the video mp4 file (or the ByteArray with the video as a result). Instead, Apple offers a default dialog to share the video to social networks. Thats why ReplayKit cannot fully replace FW for now. We’ll be offering it only as part of FW iOS ANE, for those who prefer to capture audio without FWSoundMixer and don’t mind the missing video file access (& are happy with the social sharing support).

Another big downside currently has to do with AIR. AIR doesn’t support ReplayKit framework in its latest version still. Even it is possible to workaround this on Mac, you can’t workaround that on Windows. Likely Adobe will need to rebuild their version of ld64 Apple linker, as the frameworks including ReplayKit now work with different stub formats than they used to.

We’ll see how long this takes, hopefully not too long. In the meantime it will be possible to build ReplayKit supported projects only on Mac.

Custom resolutions

As mentioned, FW iOS will start supporting custom resolutions when recording in realtime mode. Until now, you were stuck with harcoded resolution depending on what AIR requested from iOS. This was also a default when recording from highres apps. But not anymore – if your app is highres and you wish to record for example to 1920 x 1080 – or, on the other hand, you wish to make the final video really lowres (400 x 300) to send it faster over the network, you are free to do so. Just specify the dimensions as usual with setDimensions(wh) or within start and FW will scale the video as needed.

If you don’t specify any dimensions FW will behave as usual to not break backward compatibility.

FW 2.4 iOS bug (update)

EDIT: This appears to be fixed now. Re-download FW 2.4 archive with the updated iOS ANE inside. Thanks Jake, Jon & others for providing details!

There’s a bug in the 2.4 release – it is most likely connected to saveToGallery on iOS and demonstrates itself as a crash just after stopping the recording. We know about the issue and are working to fix this asap. As a workaround try to use saving to gallery with empty arguments, which won’t attemp to use albums (the likely cause of the crash):

myEncoder.saveToGallery(“”, “”);


FW 2.4 is out

FlashyWrappers 2.4 SDK has been finally released. There have been more changes since PR2, most notable probably fixing a crash affecting number of Android devices (everything with GLES3 support). Also fixed was an issue on iOS affecting highres recording.

The 2.4 SDK also includes the free FWSoundMixer 1.1, with Android & iOS ANE builds, which goes in line with the extended Android focus.

There haven’t been as many changes to the last 2.4 PR2 build, but compared to version 2.3, I think this version should be called 3.0 (well, maybe next time). Just take a look at what has changed:

  • Pretty significant API changes, with added focus on automating some annoying things and simplification. Some methods which were needed in the past(such as setFrameCount, canFinish) were wiped out. Some obscure hacks were also gotten rid of, such as in FW 2.3 you had to initialize iOS with -1 x -1 dimensions if you wanted to capture in fullscreen.
  • The division into realtime and non-realtime recording modes. In FW 2.3, both modes were mixed up, something work as realtime, something as non-realtime, depending on platform.
  • Total rewrite of the Windows platform. No more FFmpeg, FW 2.4 relies on MediaFoundation, which is native Windows API for encoding videos. This also means free MP4’s on desktop AIR.
  • Total rewrite of the Android platform. Also no more FFmpeg, which means saying goodbye to 3fps encoding. The performance still doesn’t match iOS mostly but its getting much closer than before(we are in the 20s,30s,40s in fps at least, on decent devices). Unfortunately relying on native Android API means element of instability, but we’re determined to work on that until the stability is acceptable. Oh and this rewrite also means free MP4’s on Android (before you only had OGV’s).
  • Porting FW iOS to Mac, so pretty much the same as the two above, no more FFmpeg reliance and free MP4’s for this platform.

There have been new issues popping up all the time before this release, but if we’ve waited until everything was perfect, we would never release this version(or anything in general!). At least the priorities for 2.5 have partially filled up.

Updated pricing

One thing you might have noticed is different pricing strategy. This was not an easy change but it had to be done. There are now 2 paid versions of FW SDK:

FW “Full” – includes iOS, Mac and Windows, the pricing stays ($197).

FW “Premium” – includes iOS, Mac, Windows, Flash, Android ($347).

The free version still includes all platforms, but the 30-second limit for recording videos is in all versions now (AS3 throws an Exception after 30 seconds worth of video length, indicating why it happened).

The “Premium” tier was added because of a simple reason. The Flash platform has always been the hardest and by far most time consuming platform to support. One reason is the slowness of the Flash VM and various issues / bugs / needed features stemming from that.  Another reason is the risky environment of automatic Adobe Flash updates often changing or breaking something in FlashyWrappers. Particularly this year Adobe has generated number of issues in the Flash Plugin, including slowing down Alchemy, making Alchemy not work with threads or making Flash Plugin crash when Alchemy code was used with threads (so this was not just FW issues, but all Alchemy based SWC’s). We’ve spent dozens of hours trying to debug these issues & communicating with Adobe, and unfortunately cannot rule out more of that in the future. As lesson learned, we feel the price increase for the extended Flash support is justified.

Android is a similar case due to the variabilities in the devices, even from the few reports we’ve got so far its clear it might be comparable or worse than Flash, at least in the beginning.

On the other hand, iOS, Windows and Mac are reasonable platforms to support. AIR platform means no automatic updates potentially ruining your application even if Adobe introduces bugs to AIR (they won’t show until you recompile and republish your app to the store, and you can almost always rollback to older AIR version). There is no huge device fragmentation, so once FW is tested on one iOS device its almost guaranteed to work on the rest.

Future versions focus

With the advent of video recording support on Android 5.0 and iOS 9.0 we are taking a look at where FW should be going. Among with implementing the new API’s into the upcoming FW SDK, we think the focus of this SDK should go more towards providing simple classes and / or template apps for the most frequent use cases (webcam recording, jukebox, “piano” player etc). As the recording API tech might become less valuable due to improved support from Apple and Google(we’ll still need the ANE’s of course), the support code around FW might be what will create the new value of this SDK for developers. In any case, as long as there’s demand FW will be developed & supported.

FlashyWrappers 2.4 
- FWSoundMixer: Added "recordDynamic" to mix in dynamic(continuously loading) sound such as URLStreams.
- iOS: iOS_saveToCameraroll was replaced by saveToGallery compatible with FW Android. This method now creates a new album optionally on iOS 8+. Please check the documentation.
- iOS: Fixed an issue with recording in highres on some devices (error -6683 when creating pixel buffer cache texture)
- Android: Fixed a crash on all GLES3 compatible devices. FW will properly initialize when AIR uses GLES3 as well as GLES2.
- Android: Finished implementing addAudioFrame
- Android: Optimized audio buffer queue
- Android: Added addAudioSoundtrack 
- Android: Added method for saving video to gallery
- Android: Fixed an issue which would freeze recording on second try if audio was being recorded
- Android: Ported FWSoundMixer to Android for recording dynamic AIR audio sounds, added method for converting floats to shorts on native level to make sending audio to Android encoders faster