FW is back

After a pause, FW is getting back on track.

The support is resuming and there will be some improvements and new features pretty soon, though cleaning up the code is a priority right now, before fully resuming. Also FW will be ported to Unity3D.

One of the biggest ongoing pains used to be Android, but hopefully as the phones are improving and the obscure encoders implementations on older Android versions are becoming more rare, this issue auto corrected itself significantly during the last year.

Nevertheless, Android will still remain in alpha as we’ll work on making sure it goes into beta as fast as possible.

FW 1 year licensing & changes

Hello everyone – Since last year there has been info on FW website mentioning the free support and version updates for 1 year as part of pricing. This info is not new, it has also been contained within the FW package license since FW 2.2.

So far, everyone had free access to the latest versions of FW – this will change with the next release though. For those purchasing FW 2.2 or later (released Oct 2014), the 1 year period was over few months ago.  We’ve still left offset of few months to cover for those first few months where that info might not have been apparent on the website even it was always contained inside the license.

That being said, all customers purchasing between end of Oct 2014 and end of March 2015 will need to re-purchase FW license to get access to new updates and high priority e-mail support.

All customers purchasing before Oct 2014(including), will get an additional extra year of free updates, but we won’t be able to support those customers after that period unless they re-purchase their licenses. There is no 1 year limit in the pre Oct-2014 licenses, but also no promised free updates.

We will be still supporting everyone with product-wide bugs of course, regardless if you re-purchase your license or not, as working product is in the best interest of everybody. You’ll still be able to use older FW versions without repurchasing new licenses, at your own risk however.

The info about how to login to the new update system will be emailed to all customers once the system is ready.

FW 2.55 released (mobile focused)

New version of FW was released, accumulating feedback we’ve had – this release is exclusively mobile focused and includes stability fixes / new features for both Android and iOS.

The most notable news about Android is, FW was tested on range of devices, and there has been an issue with its GL shaders discovered which prevented FW from working properly on number of devices. This has been fixed so FW should be more reliable on Android now.

While testing, there has been particular set of issues discovered, which have to do with Mediatek encoders (on devices from brands such as Meizu or Lenovo). Recording currently seems to work reliably only in 1280×720 resolution with this encoder. FW is therefore trying to detect this encoder automatically and force 1280×720 resolution. It also offers you a way to detect / react to this encoder from AS3 (maybe you’ll want to disable recording completely).

On iOS, a pretty significant memory leak has been fixed, photo permissions are now detectable and dispatched as events – for more changes and fixes see the changelog below and / or the updated API docs.

FlashyWrappers 2.55
------------------
- Android: Fixed an issue on various devices caused by shader optimizing out an attribute. The recording wouldn't work at all - the shader attributes are now assigned manually to prevent this from happening.
- Android: On devices using Mediatek encoder(such as Lenovo, Meizu), video encoding appears to work reliably only in 1280x720 with the current FW version. FW will force video dimensions to 1280x720 when this encoder is detected. You can adjust your video frames dimensions if needed, by using "if (myEncoder.getAndroidEncoderQuirks() == "quirk_mtk") {...}".
- Android: Fixed an issue with 2nd recording with microphone failing on some devices by completely releasing AudioRecorder instead of just stopping it.                                                                                                                                                                                                  
- Android / iOS: The video filename can be now modified easily on both platforms by setting myEncoder.mobileFilename = 'somefile.mp4';
- All platforms: FW now checks if the audio data is empty or not to prevent unexpected errors when sending audio data with zero length.
- All platforms: FW now checks if the ByteArray isn't overflowing in addVideoFrame to avoid subsequent issues in the encoders.
- All platforms: addSoundtrack on Android is being added at the start in non-realtime mode.
- iOS: Killed the video composing in final video recording phase, this significantly shortens the final phase time and also allows to finish the recording even when the app is suspended.
- iOS: As a result of the change above, the video bitrate setting has effect in realtime mode too (before it had effect only in non-realtime mode).
- iOS: Fixed a memory leak, after each recording approx +6-10MB of memory was consumed by texture cache which wasn't cleared properly. This could have resulted in low memory warnings and application shutdown if recording many times in row.
- iOS: Video foler is marked as preventBackup = true, to prevent backing it up to the iCloud.
- iOS: Added photo gallery permissions status events to allow you reading those permissions and displaying the permissions popup earlier than just before the video is trying to get saved. For more see API docs.

FW 2.5 released – 64bit Mac, API docs & rectangles

Hi everyone, new version of FlashyWrappers SDK is out. Apart from more fixes, the most notable new feature of this version allows you to capture rectangular video areas on iOS and Android. It also adds 64-bit support to OS X and first iteration of the on-line API docs.

64-bit OS X support

The 2.5 release also includes the 64-bit ANE build for Macs. It still includes 32-bit ANE build but this will most likely be discontinued unless there is a need to fallback to older AIR versions because of some issues in AIR 20.

Rectangular capture

This feature works in two modes and each method has its advantages and drawbacks.

1) Calculate

This method assumes your stage scale mode is set to NO_SCALE, and your stage alignment is TOP_LEFT. That is the biggest drawback of this method – however if you meet these criteria(should be true for all Starling and similar apps), this method should capture the exact area you specify.

2) Visual

This method doesn’t mind which stage scale or alignment mode is set. It places visual markers (2×2 mini rectangle), of the color you choose (it must be unique for at least couple of frames – up to 10 to be safe), at 2 corners of the capture area. This allows FW to measure the actual OpenGL coordinates despite all of the scaling / shifting of stage AIR might be doing, depending on the device, your stage size and so on.

Apart from plotting the markers for a short time, other drawbacks currently include the possibility of the markers deforming after AIR scales itself and the masurement not going exact – so you should always keep offsets around the capture area when using this method.

Note that there are some simple precautions done to prevent bad marker reading if its too scaled up or down, but it wasn’t fully tested on all possible devices so use the visual method with caution.

API docs

You can access the API docs online at http://flashywrappers.com/asdoc/  (or through the standard menu on the main website). This is automatically generated and it will be updated on each new release.

The API docs will hopefully fill in the blanks into the PDF docs, which are more explanatory and short examples but do not cover 100% of the API.

Full changelog:

FlashyWrappers 2.5
------------------
- All platforms: addSoundtrack now generates a warning when used in realtime mode. Using this method in realtime WILL cause desync. For any realtime & synced audio you should use FWSoundMixer. - DONE
- All platforms: setRealtime method added to avoid the need to set it in the start method.
- All platforms: The comments in AS3 wrapper were rewritten to be compatible with automatic documentation system & API docs added.
- All platforms: Added checks for dimensions being a multiply of 16, warning will be generated if it is not.
- All platforms: Added checks for valid H.264 resolutions, error will be thrown if the resolution is wrong.
- All platforms: Added checks for valid AAC bitrates, error will be thrown if the bitrate is wrong.
- Mac: Added OS X 64-bit ANE version for AIR 20+. The 32-bit ANE will be still kept in this release, for compatibility reasons, but beginning next release it won't be.
- Windows: Fixed a crash caused by addSoundtrack on Windows 7.
- Windows: Code was completely refactored.
- iOS: Fixed recording capturing black screen only on iOS (this was happening on random devices) with depthAndStencil and high resolution both enabled.
- Android: setDimensions now forces video resolution in relatime mode, exactly like on iOS. 
- iOS / Android: The "default" library was erased from iOS. For testing on desktop simply use FW for desktop.
- iOS / Android: Added setCaptureRectangle for mobile to capture rectangular area in two modes (visual and calculation method). See API docs for more details.

Coming up in 2016…

FW_API

 

It’s the complete API doc, both offline and online – coming up soon!

Some of the other changes coming up:

– Capture rectangular area of target DisplayObject on mobile in OpenGL mode (to workaround the unability to capture DisplayObjects in this mode).

– API giving you better hints to avoid non-sensical or dangerous parameters when initializing the encoder.

– Video replay classes for desktop & mobile.

– Implementing new video recording API’s for mobile allowing you to leverage the latest in mobile video recording on iOS / Android while still supporting the low-level video recording API’s.

– Better audio support through FWSoundMixer & addAudioSoundtrack

– Improving Android.

And more…

Happy New Year 2016!

All the best with your projects for the next year to everyone, especially all of our  FlashyWrappers customers!

Thanks to your support FlashyWrappers is alive and well, and will continue so through to the next year. Apologies if the work has been a bit slower these last two weeks – the Christmas period had its impact.

Rest assured there’s already a changelog for FW 2.5 existing, with 19 pre-written changes / fixes / new features which should make it in the next release. Also keep in mind we do listen to your feedback and sometimes the new features we’re adding come from merely you asking questions. For example, there has been a question / request if the FWSoundMixer implements only some of the SoundMixer / Sound methods – and it really doesn’t implement almost anything. Seeing how difficult it is with enabling native audio recording we’re thinking that it would be actually a good focus to instead finish FWSoundMixer properly by adding some more important methods to control the volume for example. Native audio recording is still on our list but seeing its not what most of our customers need (usually microphone or soundmixer are enough), it’s not as high priority as it used to be.  So here you can see just one example of how even your questions can affect our thinking / focus.

Granted, sometimes we need to push your requests or questions for later as there might be higher priority work to be done – its nothing personal though, many times its simply because more people request something else, or something more serious is needed for the product in the big picture – or simply, it doesn’t align with the general direction and goals we see as a good trend for our SDK.

The next release is targeted at sometime in January, and will be focused on further tightening the reliability, ease of use and getting rid some of the known frustrations. Also we’ll be on the lookout to improve Android support all the time.  The Mac / iOS is about to get internal API overhaul too, similar to what Android got this / last month (you won’t see much of that externally though).

Overall, we’ll try to bring much more polished experience and make the SDK the best possible video encoding SDK for AIR / Flash possible. Hopefully, by the end of next year there will be nothing major to add or change and we can just expand on extra features such as integrated video replays and seamless support for the new video API’s.

See you in 2016!

 

FW 2.45 released

The latest release fixes some issues found by customers in the 2.4 release. It also features refactored Android ANE. All customers can use the links they used for purchasing to obtain v2.45.

A bit of non-feature at first: We’ve added full ReplayKit support for iOS, but found out AIR for Windows still doesn’t support iOS9 frameworks so decided not to add the AS3 endpoints (otherwise Windows users would be unable to compile with the new iOS ANE).

One of the more notable features which were added in this release are a new audio recording mode in Flash trying to address desync, which happens only in Flash and only on some hardware based on what we’ve heard. As this appears hardware / Flash related it might not fix the issue but the latest known “pain” of the realtime Flash recorder was the audio. On all other platforms, the audio is using system time for its timestamps, but in Flash this was causing issues, as the Flash timer is not accurate and basically depends on fps. That’s why the Flash recorder was using a mix of realtime timestamps for video but monotonic timestamps for audio. That might have been a cause of audio desync under some conditions where audio starts arriving too early / late. The new experimental recording mode still uses monotonic timestamps but also compares with system time and when the delta is too high it corrects the monotonic timestamps to prevent them drifting apart too much.

In case the desync issue is in how Flash works with the microphone on a given platform, this won’t fix anything unfortunately(there is no way FW can “inspect” the audio data and determine if its content is too “late” or “early” – its just bunch of numbers to FW). It can fix the case where the audio as a whole was starting to come in too late / early since the command to record was given.

A new iOS feature which should be ported to Android in the next release is being able to set any video resolution, in realtime mode. Until now, the video resolution was hardcoded to whatever AIR asked iOS to use for your app (on iPad typically 1024×768). In case of high resolution (2048×1536 in this case) however, the videos were still recorded at 1024×768 by FW. This is no longer the case and the videos will be recorded at 2048×1536. Plus, you can now override the resolution with setDimensions like in non-realtime. You can use almost any resolution – iOS might not like certain dimensions and refuse to encode or save to camera roll. To be safe make sure your dimensions can be divided by 16.

This release was otherwise mostly about maintenance after the exhausting 3-phase 2.4 release. We’ll still do that during 2016 of course plus finally get to add support classes for AIR, such as a class for easily recording a webcam on all platforms or replaying recorded videos. We’ll also finally join all the platforms into a single ANE (that is an understandable request we’ve heard several times). Mac 64 bit support is also high in priorities(we are aware of this for AIR 20+). More work on Android is expected as well. As always let us know what you personally would like to see in the following releases –

The v2.45 changelog:

- Flash: Added a new, more robust audio recording mode, which uses both "assumed" and "realtime" timestamps to determine the pts. This is an experimental mode trying to get rid of random audio desync reported in Flash. To use the mode call myEncoder.setAudioRealtime(true).
- Flash: Fixed too early initialization issue both in realtime and non-realtime mode, resulting in number of other issues, including audio soundtrack not being added or other unpredictable behavior.
- Flash (MP4's only): After encoding, the moov atom in mp4's is being moved to the start now, allowing the recorded videos to be streamable (using ffmpeg's "faststart" option).
- Mac / Flash: (non-realtime) Fixed addSoundtrack issue with calculating pts resulting in the audio tack not being hearable.             
- iOS: Implemented the new ReplayKit framework, unfortunately because AIR for Windows doesn't support it yet we didn't enable it in this version(otherwise you would be unable to build any apps on Windows AIR with our ANE).
- iOS: Fixed a bug with ANE reporting "stopped" event even when the app was minimized but no recording was going on.
- iOS: iOS realtime mode respects resolution now. If you publish your AIR project in highres, normally it was scaled down(for example 2048x1536 was scaled down to 1024x768 video). Now, the video will be truly recorded at 2048x1536 - you can still force downscaling however through setDimensions(1024x768).
- iOS: saveToGallery doesn't create the default "FlashyWrappers" album anymore, when no arguments are specified in saveToGallery. It behaves like iOS_saveToCameraRoll did now instead without arguments
- Windows: better error logging added to addVideoFrame based on some reports with Windows encoder issues
- Android: refactored all of the source code into new classes, moved the video recorder into its own GLES context to avoid switching contexts.
- Android: errors are detected when starting FW and on any error FW forces the recording to not continue (it returns instantly from the FW recording method if you're still trying to call it). This is trying to prevent the app from breaking while recording and also for a cleaner exit.
- Android: saveToGallery without arguments will use "Videos" as default album name now
- Android: fixed a freeze which would sometimes happen when finishing audio thread because there were no input buffers available and FW would way indefinitely for a free buffer, to send the "finishing" audio data. Additional drain was added to fix the issue.

iOS: Upcoming ReplayKit support & custom resolutions

Even the main focus is Android these days, we’re not forgetting iOS either. In iOS 9, a new video recording API was introduced, called ReplayKit. We are preparing support for this API. The main advantage, besides the ease of use when implementing the native code (which is not a concern for AIR devs anyway), is full audio recording support (at least if the videos we saw don’t lie). It should be possible to include microphone as well, but otherwise what the game is playing out of speakers appears to be recorded in the video.

ReplayKit is completely separate from the FW video capture path which uses AVFoundation & OpenGL ES, so there are some disadvantages as well. You won’t be able to configure the video quality or resolution (this is also an upcoming feature of the next FW version, see below).

The biggest downside however appers to be the inability to access the video mp4 file (or the ByteArray with the video as a result). Instead, Apple offers a default dialog to share the video to social networks. Thats why ReplayKit cannot fully replace FW for now. We’ll be offering it only as part of FW iOS ANE, for those who prefer to capture audio without FWSoundMixer and don’t mind the missing video file access (& are happy with the social sharing support).

Another big downside currently has to do with AIR. AIR doesn’t support ReplayKit framework in its latest version still. Even it is possible to workaround this on Mac, you can’t workaround that on Windows. Likely Adobe will need to rebuild their version of ld64 Apple linker, as the frameworks including ReplayKit now work with different stub formats than they used to.

We’ll see how long this takes, hopefully not too long. In the meantime it will be possible to build ReplayKit supported projects only on Mac.

Custom resolutions

As mentioned, FW iOS will start supporting custom resolutions when recording in realtime mode. Until now, you were stuck with harcoded resolution depending on what AIR requested from iOS. This was also a default when recording from highres apps. But not anymore – if your app is highres and you wish to record for example to 1920 x 1080 – or, on the other hand, you wish to make the final video really lowres (400 x 300) to send it faster over the network, you are free to do so. Just specify the dimensions as usual with setDimensions(wh) or within start and FW will scale the video as needed.

If you don’t specify any dimensions FW will behave as usual to not break backward compatibility.

FW 2.4 iOS bug (update)

EDIT: This appears to be fixed now. Re-download FW 2.4 archive with the updated iOS ANE inside. Thanks Jake, Jon & others for providing details!

There’s a bug in the 2.4 release – it is most likely connected to saveToGallery on iOS and demonstrates itself as a crash just after stopping the recording. We know about the issue and are working to fix this asap. As a workaround try to use saving to gallery with empty arguments, which won’t attemp to use albums (the likely cause of the crash):

myEncoder.saveToGallery(“”, “”);

 

FW 2.4 is out

FlashyWrappers 2.4 SDK has been finally released. There have been more changes since PR2, most notable probably fixing a crash affecting number of Android devices (everything with GLES3 support). Also fixed was an issue on iOS affecting highres recording.

The 2.4 SDK also includes the free FWSoundMixer 1.1, with Android & iOS ANE builds, which goes in line with the extended Android focus.

There haven’t been as many changes to the last 2.4 PR2 build, but compared to version 2.3, I think this version should be called 3.0 (well, maybe next time). Just take a look at what has changed:

  • Pretty significant API changes, with added focus on automating some annoying things and simplification. Some methods which were needed in the past(such as setFrameCount, canFinish) were wiped out. Some obscure hacks were also gotten rid of, such as in FW 2.3 you had to initialize iOS with -1 x -1 dimensions if you wanted to capture in fullscreen.
  • The division into realtime and non-realtime recording modes. In FW 2.3, both modes were mixed up, something work as realtime, something as non-realtime, depending on platform.
  • Total rewrite of the Windows platform. No more FFmpeg, FW 2.4 relies on MediaFoundation, which is native Windows API for encoding videos. This also means free MP4’s on desktop AIR.
  • Total rewrite of the Android platform. Also no more FFmpeg, which means saying goodbye to 3fps encoding. The performance still doesn’t match iOS mostly but its getting much closer than before(we are in the 20s,30s,40s in fps at least, on decent devices). Unfortunately relying on native Android API means element of instability, but we’re determined to work on that until the stability is acceptable. Oh and this rewrite also means free MP4’s on Android (before you only had OGV’s).
  • Porting FW iOS to Mac, so pretty much the same as the two above, no more FFmpeg reliance and free MP4’s for this platform.

There have been new issues popping up all the time before this release, but if we’ve waited until everything was perfect, we would never release this version(or anything in general!). At least the priorities for 2.5 have partially filled up.

Updated pricing

One thing you might have noticed is different pricing strategy. This was not an easy change but it had to be done. There are now 2 paid versions of FW SDK:

FW “Full” – includes iOS, Mac and Windows, the pricing stays ($197).

FW “Premium” – includes iOS, Mac, Windows, Flash, Android ($347).

The free version still includes all platforms, but the 30-second limit for recording videos is in all versions now (AS3 throws an Exception after 30 seconds worth of video length, indicating why it happened).

The “Premium” tier was added because of a simple reason. The Flash platform has always been the hardest and by far most time consuming platform to support. One reason is the slowness of the Flash VM and various issues / bugs / needed features stemming from that.  Another reason is the risky environment of automatic Adobe Flash updates often changing or breaking something in FlashyWrappers. Particularly this year Adobe has generated number of issues in the Flash Plugin, including slowing down Alchemy, making Alchemy not work with threads or making Flash Plugin crash when Alchemy code was used with threads (so this was not just FW issues, but all Alchemy based SWC’s). We’ve spent dozens of hours trying to debug these issues & communicating with Adobe, and unfortunately cannot rule out more of that in the future. As lesson learned, we feel the price increase for the extended Flash support is justified.

Android is a similar case due to the variabilities in the devices, even from the few reports we’ve got so far its clear it might be comparable or worse than Flash, at least in the beginning.

On the other hand, iOS, Windows and Mac are reasonable platforms to support. AIR platform means no automatic updates potentially ruining your application even if Adobe introduces bugs to AIR (they won’t show until you recompile and republish your app to the store, and you can almost always rollback to older AIR version). There is no huge device fragmentation, so once FW is tested on one iOS device its almost guaranteed to work on the rest.

Future versions focus

With the advent of video recording support on Android 5.0 and iOS 9.0 we are taking a look at where FW should be going. Among with implementing the new API’s into the upcoming FW SDK, we think the focus of this SDK should go more towards providing simple classes and / or template apps for the most frequent use cases (webcam recording, jukebox, “piano” player etc). As the recording API tech might become less valuable due to improved support from Apple and Google(we’ll still need the ANE’s of course), the support code around FW might be what will create the new value of this SDK for developers. In any case, as long as there’s demand FW will be developed & supported.

FlashyWrappers 2.4 
------------------
- FWSoundMixer: Added "recordDynamic" to mix in dynamic(continuously loading) sound such as URLStreams.
- iOS: iOS_saveToCameraroll was replaced by saveToGallery compatible with FW Android. This method now creates a new album optionally on iOS 8+. Please check the documentation.
- iOS: Fixed an issue with recording in highres on some devices (error -6683 when creating pixel buffer cache texture)
- Android: Fixed a crash on all GLES3 compatible devices. FW will properly initialize when AIR uses GLES3 as well as GLES2.
- Android: Finished implementing addAudioFrame
- Android: Optimized audio buffer queue
- Android: Added addAudioSoundtrack 
- Android: Added method for saving video to gallery
- Android: Fixed an issue which would freeze recording on second try if audio was being recorded
- Android: Ported FWSoundMixer to Android for recording dynamic AIR audio sounds, added method for converting floats to shorts on native level to make sending audio to Android encoders faster