FW 2.4 iOS bug (update)

EDIT: This appears to be fixed now. Re-download FW 2.4 archive with the updated iOS ANE inside. Thanks Jake, Jon & others for providing details!

There’s a bug in the 2.4 release – it is most likely connected to saveToGallery on iOS and demonstrates itself as a crash just after stopping the recording. We know about the issue and are working to fix this asap. As a workaround try to use saving to gallery with empty arguments, which won’t attemp to use albums (the likely cause of the crash):

myEncoder.saveToGallery(“”, “”);

 

FW 2.4 is out

FlashyWrappers 2.4 SDK has been finally released. There have been more changes since PR2, most notable probably fixing a crash affecting number of Android devices (everything with GLES3 support). Also fixed was an issue on iOS affecting highres recording.

The 2.4 SDK also includes the free FWSoundMixer 1.1, with Android & iOS ANE builds, which goes in line with the extended Android focus.

There haven’t been as many changes to the last 2.4 PR2 build, but compared to version 2.3, I think this version should be called 3.0 (well, maybe next time). Just take a look at what has changed:

  • Pretty significant API changes, with added focus on automating some annoying things and simplification. Some methods which were needed in the past(such as setFrameCount, canFinish) were wiped out. Some obscure hacks were also gotten rid of, such as in FW 2.3 you had to initialize iOS with -1 x -1 dimensions if you wanted to capture in fullscreen.
  • The division into realtime and non-realtime recording modes. In FW 2.3, both modes were mixed up, something work as realtime, something as non-realtime, depending on platform.
  • Total rewrite of the Windows platform. No more FFmpeg, FW 2.4 relies on MediaFoundation, which is native Windows API for encoding videos. This also means free MP4’s on desktop AIR.
  • Total rewrite of the Android platform. Also no more FFmpeg, which means saying goodbye to 3fps encoding. The performance still doesn’t match iOS mostly but its getting much closer than before(we are in the 20s,30s,40s in fps at least, on decent devices). Unfortunately relying on native Android API means element of instability, but we’re determined to work on that until the stability is acceptable. Oh and this rewrite also means free MP4’s on Android (before you only had OGV’s).
  • Porting FW iOS to Mac, so pretty much the same as the two above, no more FFmpeg reliance and free MP4’s for this platform.

There have been new issues popping up all the time before this release, but if we’ve waited until everything was perfect, we would never release this version(or anything in general!). At least the priorities for 2.5 have partially filled up.

Updated pricing

One thing you might have noticed is different pricing strategy. This was not an easy change but it had to be done. There are now 2 paid versions of FW SDK:

FW “Full” – includes iOS, Mac and Windows, the pricing stays ($197).

FW “Premium” – includes iOS, Mac, Windows, Flash, Android ($347).

The free version still includes all platforms, but the 30-second limit for recording videos is in all versions now (AS3 throws an Exception after 30 seconds worth of video length, indicating why it happened).

The “Premium” tier was added because of a simple reason. The Flash platform has always been the hardest and by far most time consuming platform to support. One reason is the slowness of the Flash VM and various issues / bugs / needed features stemming from that.  Another reason is the risky environment of automatic Adobe Flash updates often changing or breaking something in FlashyWrappers. Particularly this year Adobe has generated number of issues in the Flash Plugin, including slowing down Alchemy, making Alchemy not work with threads or making Flash Plugin crash when Alchemy code was used with threads (so this was not just FW issues, but all Alchemy based SWC’s). We’ve spent dozens of hours trying to debug these issues & communicating with Adobe, and unfortunately cannot rule out more of that in the future. As lesson learned, we feel the price increase for the extended Flash support is justified.

Android is a similar case due to the variabilities in the devices, even from the few reports we’ve got so far its clear it might be comparable or worse than Flash, at least in the beginning.

On the other hand, iOS, Windows and Mac are reasonable platforms to support. AIR platform means no automatic updates potentially ruining your application even if Adobe introduces bugs to AIR (they won’t show until you recompile and republish your app to the store, and you can almost always rollback to older AIR version). There is no huge device fragmentation, so once FW is tested on one iOS device its almost guaranteed to work on the rest.

Future versions focus

With the advent of video recording support on Android 5.0 and iOS 9.0 we are taking a look at where FW should be going. Among with implementing the new API’s into the upcoming FW SDK, we think the focus of this SDK should go more towards providing simple classes and / or template apps for the most frequent use cases (webcam recording, jukebox, “piano” player etc). As the recording API tech might become less valuable due to improved support from Apple and Google(we’ll still need the ANE’s of course), the support code around FW might be what will create the new value of this SDK for developers. In any case, as long as there’s demand FW will be developed & supported.

FlashyWrappers 2.4 
------------------
- FWSoundMixer: Added "recordDynamic" to mix in dynamic(continuously loading) sound such as URLStreams.
- iOS: iOS_saveToCameraroll was replaced by saveToGallery compatible with FW Android. This method now creates a new album optionally on iOS 8+. Please check the documentation.
- iOS: Fixed an issue with recording in highres on some devices (error -6683 when creating pixel buffer cache texture)
- Android: Fixed a crash on all GLES3 compatible devices. FW will properly initialize when AIR uses GLES3 as well as GLES2.
- Android: Finished implementing addAudioFrame
- Android: Optimized audio buffer queue
- Android: Added addAudioSoundtrack 
- Android: Added method for saving video to gallery
- Android: Fixed an issue which would freeze recording on second try if audio was being recorded
- Android: Ported FWSoundMixer to Android for recording dynamic AIR audio sounds, added method for converting floats to shorts on native level to make sending audio to Android encoders faster

FW 2.4 update 2

It didn’t take too long and there’s more news!

First of all, there were more fixes done on FW 2.4 Android. There was one nasty issue which prevented addAudioFrame from working. That has been corrected, and FW is now able to record audio in both realtime and non-realtime mode (at least on our devices). Non-realtime mode is useful for anything like composing video + audio together as “final step”, and realtime is well, realtime. It just records whatever audio is currently being sent to FW (typical use case is recording a gameplay video). Realtime mode uses system time for timestamps of each audio frame.

With that in mind, we’ve ported FWSoundMixer to Android. As a result, you’ll be able to record dynamic audio your app is generating together with the video, just like on iOS. This was tested on the Hungry Hero sample game, and I wish I could share a video of that recording but sadly after upload to YouTube some strange lags have appeared in the audio(they are not showing up in the original video). It might have to do something with pts and the way YouTube interprets them, in any case we’ll take a bit more look at that to make sure there’s a “corrected” video available for the final 2.4 release.

This was the last feature needed for the 2.4 release. Feature-wise, Android is now almost identical to iOS. There will be still work going on with everything else(the website will go through some changes as well) and lots to be done in terms of stability, but thats the upcoming chapter. For now, this alpha is good enough for us to not delay the final FW 2.4 SDK release any longer.

 

FW 2.4 update

Progress report on the 2.4 final release:

In the new Android, addSoundtrack was finally implemented – this involved some extra threading and reworking the audio code in general. In any case you’ll be able to finally add background soundtracks to your “non-realtime” videos. Working on retesting addAudioFrame currently.

Saving to gallery was also implemented for Android, mirroring & improving  FW’s current save to camera roll functionality on iOS. You’ll be able to name the gallery album when saving your video and also the filename like this:

myEncoder.saveToGallery(“My cool app”, “video_” + dateAndTime + “.mp4”);

This method is multiplatform in 2.4 final and on iOS, it replaces iOS_saveToCameraRoll. On iOS, this updated method will also create a custom album if all works well.

Finally, the last bit of iOS news: a nasty issue was fixed when trying to record in high resolution(thanks Jake!). This issue was triggering on some devices while on other devices the recording was working prefectly fine. If you had any issues with highres recording from AIR, FW 2.4 final release should fix your issues.

FlashyWrappers 2.4 PR2 available for download

As promised couple of days ago – download link below:

http://flashywrappers.com/sendlink?product=2

Please note that this is still prerelease mostly due to Android at this point, the rest of platforms were tested & gotten rid of most serious issues reported by various users – those issues mostly popped up with the new 2.4 API. Of course there are still issues expected & known.

The next 2.4 release is going to be standard (no more pre-releases). Android will stay in alpha though until we manage to stabilize even more & have enough proof its working reliably accross variety of devices. We’ll provide standard support for Android but including this in your product will be still at your own risk. At this point we’re looking forward working with anyone interested to include FW Android ANE in testing it & giving us feedback to fix issues.

Whats planned for the final 2.4 release?

  • Android – stabilizing audio & making grabbing GLES context more reliable.
  • Android – better error handling & start work on graceful exits, handle app suspension.
  • Android – whatever issues pile up within the next ~2 weeks.
  • iOS & Android – Fixing progress bar to not show 100% all the time but reflect the actual progress a bit better.
  • iOS – addSoundtrack ability to supply custom track time (in ms).
  • iOS – won’t be included in 2.4 but will begin to see how to implement the new native iOS9 video recording API.
  • There won’t be many changes in 2.4, which will be mostly Android focused again.
  • Website overhaul: Updated roadmap, pricing, features, adding support comments / forum.

Changelog for PR2 release:

 

FlashyWrappers 2.4 Prerelease 2 (unofficial)                                                                       
--------------------------------------------
- !! All platforms: start was made async. As a result of that, you need to implement "started" status event. You need to catch it after start has been called and move any code which you called after start into this code block. In Flash, making start async helps to reduce the startup lag by ~80%.
- All platforms: Fixed various bugs involving things going wrong when re-initializing the recordings (connected to the changes in 2.3->2.4 API)
- All platforms: Added better logging & ability to influence logging levels (LOGGING_BASIC or LOGGING_VERBOSE) for improved debugging (still working on this).
- All platforms: Added the option to override PTS calculation mode (forcePTSMode) with the option to specify either PTS_AUTO, PTS_MONO or PTS_REALTIME. This allows to mix realtime mode features(OpenGL fullscreen capture, multithreading) with non-realtime mode or vice versa.
- All platforms: Added the option to override frame dropping mode (forceFramedropMode) with the option to specify FRAMEDROP_AUTO, FRAMEDROP_OFF or FRAMEDROP_ON. This allows to mix realtime mode features(OpenGL fullscreen capture, multithreading) with non-realtime mode or vice versa.
- Windows: Fixed a bug which caused the new MediaFoundation based Windows ANE crashing on Windows 7.
- Windows: Added better error handling and logging when unexpected error happens.
- Android: Added capturing individual DisplayObjects in the new FW Android. You can also use addVideoFrame with ByteArray.
- Android: Added non-realtime mode in the new FW Android. This influences the pts calculations and frame dropping.
- Android: Added logging to separate log file on sd card, and verbose mode for more detailed logging.
- Android: Added AUDIO_MICROPHONE audio mode. This mode uses Android native microphone recording (not AIR microphone recording as on other platforms) for faster and reliable microphone & video recording. 
- Android: Added AUDIO_MONO and AUDIO_STEREO. Beware, these are experimental. When adding audio don't add buffers larger than 32768 bytes! Currently, when adding audio using these modes the app might freeze(its a known issue with audio).
- Android: Rewritten code extensively to support draining the audio & video encoders in separate threads (ie. improved overall performance)
- Android: Fixed Android 4.4+ issue - the video file is by default saved to apps folder on sd card now instead of the device internal storage.
- Android: Improved error handling, many Java errors(including unexpected errors) will now propagate to AS3 console in addition to the device log, making it easier to spot more serious issues instantly.
- Android: The ANE class wrapper was united with the rest of the platforms, so internally they are all using the same code now (previously Android used separate class).
- Android: FW FFmpeg Android is not being supported anymore. It will be still provided upon request (with FW 2.3 API).
- Flash: The domain memory is being reset after unloading the FW SWF. You need to "load" the SWFBridge with each recording for this to work(it automatically unloads the previously loaded SWFBridge including clearing all the domain memory). This prevents any possible memory leaks / growing memory usage caused by Crossbridge & Flash memory fragmentation.
- Flash: Fixed a bug which didn't reset multithreading to false when initializing the encoder for second time with realtime = false after realtime = true.
- Flash: Added "RAM guard", which is able to modify the recording fps dynamically depending on whenever the encoder is keeping up with the frames being sent to prevent accumulation of raw frames in RAM.
- Flash: Added the ability to control how much domain memory will be preallocated when initializing FW (in getInstance as second argument, in AIR this doesn't affect anything).

FW 2.4 PR2 almost there

Right now the final testing & packaging of FW 2.4 PR2 is happening. Before the final 2.4 release there will be some changes on the website as well, so FW 2.4 PR2 will happen in the meanime to not further delay this new update.

Of course, there are still more fixes we’d like to get into the final 2.4 for Android (the other platforms are in much better shape).

The FW 2.4 PR2 release is now expected next week.

The delay in release was mostly caused by string of Flash Plugin issues which culminated in the Flash Plugin crashing when running Alchemy code(we were  trying to find workarounds). The second reason of the delays was trying to get to a state where the new FW Android could be released at least as “alpha” instead of “tech demo” it was till now. In the end we had to be satisfied with a state where there are number of issues but at least we understand some of the biggest issues we saw. As a result, we know some workarounds as well.

One example of such obscure issue on Android & AIR: if you initiate webcam and start recording right afterwards, everything will just slowly disintegrate. This is due to the fact that even AIR reports the webcam as initialized, at least on our device it was still initializing, and playing with GLES context in the encoder init seemed to throw the whole GLES off. A simple (pretty ugly) solution for now is always making sure to initialize webcam at least few seconds before allowing any video recording. We expect more “golden gems” like that to popup on Android / AIR unfortunately.

There are ongoing issues with audio, but at least we’ve got the microphone recording working using native code and also both non-realtime mode and capturing individual DisplayObjects is supported (this is new). Which makes this version very close to the iOS version in terms of features – not in terms of stability yet, but we’ll get at least somewhere at 75-80% stability of iOS in the end (not very optimistic I know, but thats the nature of Android). We’ll have to work more on workarounds & detecting issues and gracefully handling them to avoid crashes or freezes. We will be getting into that phase pretty soon(after dealing with addSoundtrack and making addAudioFrame & microphone more stable for the final 2.4 release – which will finish the new Android version feature-wise).

There has been also news about the video recording capability in iOS9 – you can be sure we’ll implement this in FlashyWrappers as soon as possible. Similar API should be present in Android 5.0 and we’ll be looking at that as well. We’ll be always trying to implement the latest of mobile video recording API’s in case it helps with stability / performance. And these might help with both, including native audio recording. Of course those API’s will only capture fullscreen(or maybe screen regions) – luckily FW still supports individual DisplayObject / ByteArray capturing (now also on Android).

Anyway, stay tuned for more news – this past one-two months have been pretty hard in terms of accumulated bugs and delays, but we’re marching on to make FW the most stable & feature rich video recording SDK for Flash & AIR.

Oh, and one last thing. When contacting us, please make sure your inbox is working 🙂 If we are not answering within 2-3 days latest(usually we reply sooner) it is not because we’d like to ignore people, but because we can’t get to you.  This is specifically for “CPH” now (but there have been others in the past), there’s no way to send e-mails to you – we’ve tried several e-mail accounts.

We’re looking to add support forums / comments after FW 2.4 is out to make sure there are no issues with e-mails in the future.

FW Flash instabilities after the latest FP update

Just to confirm we know about this issue and are currently working with Adobe to fix – it’s high priority for them at this point.

The issue started appearing with 18.0.0.232 Flash update (and is further present in FP 19 beta). Unfortunately it causes apps running with FW to crash the whole Flash plugin, on OS X in particular (but we had a report from Windows as well).

We’ve been working hard since last Friday to provide a minimum crashing example to Adobe to accelerate the debug proccess – that example was sent yesterday and we’re still actively looking for any kind of workaround. Note that the crashing example does not contain any FlashyWrappers code. Which means it confirms, those crashes are not caused by FW in the first place (even if there was a faulty code though, Flash should not crash). So far the crashing seems to cease or at least minimize itself if FW is in non-realtime mode (or not using multithreading in 2.3). Also no crashes for earlier Flash Player versions.

This issue is pretty big so as you can imagine it has delayed the next FW 2.4 release until this is 100% resolved & confirmed to be resolved by our customers.

The bug should not affect AIR as it seems connected to Crossbridge (Alchemy)  and Adobe’s changes to that in 18.0.0.232, where they were fixing other Alchemy related issues. Crossbridge is a cross-compiler used to build FW for Flash.

FW Android supports microphone recording & release soon

The next release of FW 2.4 is almost ready. The biggest blocker was the new FW for Android. Finally, the situation has improved as we’ve met our goal to reintroduce audio. We’ve managed to add audio recording from microphone and keep pretty solid fps (40-47 in 60fps app). The microphone recording is 100% native and seems much faster than the alternative of sending audio frames from AIR as on the other platforms (we’re still trying to find out why Android hates this method – it causes lags and freezes). FW gained about 10 fps just by going native with microphone recording.

In any case, to use this is again very simple & same as on the other platforms, just start FW with AUDIO_MICROPHONE mode.

The code was also refactored, taking advantage of multithreading when working with both audio and video encoders. The biggest thing thats left for Android is adding “non-realtime” mode support & fully resolving the audio. Also note that its still “alpha”, recommended to test but not into full release (we’ll be happy to work with you on stabilizing Android on your test devices though).

There are small changes on other platforms and to the API as well. Most notably “started” status event was added because start was made async. This eliminated  some lag start was causing in Flash – most of it is executed on background thread now.

So to summarize, whats left is the usual finishing works(pretty much packaging everything up, update the docs, examples etc.) which will still take few days, but all the features for this release are closed!

FW 2.4 Prerelease 2 info

After some initial feedback from the first prerelease, prerelease 2 is on its way.  This will very likely be the last one before the final release. We’ve fixed the new Windows ANE’s crash on Windows 7 and added better logging to all FW platforms (logging level can also be set to verbose in prerelase 2). There will be an updated documentation describing how to do this. We have plans to  improve the logging even more, but this is the first step to give users control over logging levels.

Another change coming by popular demand is the ability to force parts of FW behavior. In particular, the newly introduced realtime and non-realtime modes automatically switch various behaviors of FW. Most users probably won’t mind and will appreciate the simplicity compared to FW 2.3, but for advanced users this might not be great.

So we’ve made a compromise – the realtime and non-realtime mode of course stays, but you can now also force (non)realtime PTS calculations method or force framedropping off. This can be useful in situations such as:

  • In Flash version, you want non-realtime capture but still want to use multithreading (solution = capture with realtime mode on to use MT and force PTS to non realtime, framedrop off)
  • On mobile, you want non-realtime capture but still want to take advantage of the OpenGL accelerated capture mode, which works only when capturing in realtime mode. (solution same as above).

The current focus before releasing FW2.4 Prerelease 2 is Android – getting the audio to work again and possibly increasing the overall performance.

Even after Android is a bit more stabilized, that won’t be “it”. There’s still so much work & improvements to do, including better / faster capture methods (particularly on mobile but even on desktop). So don’t think we’re satisfied with FlashyWrappers at this point.

Just to name a few other things on our TODO list: native audio capture, video replay component, Android fallback mechanisms, API improvements, custom classes for simple FW integration (one-liner webcam recording class) and of course stability and less bugs – this should improve all the time.