FW 2.4 preview

FlashyWrappers v2.4 is coming up and here are some more details!

As you might have noticed, the versioning has accelerated a bit from 2.3 straight to 2.4. That’s because we’ve finally got a chance to get back to the general API and it is being simplified & cleared up.  Also some long delayed features have been  added to 2.4, so we felt it deserved that .4 mark.

The biggest news is the new Mac ANE, which comes just after the new Android ANE introduced in the previous release. This ANE is not “alpha” because it’s 90% identical to iOS ANE, which was already tested by our customers. This ANE outputs mp4 videos natively  & uses AVFoundation instead of FFmpeg, so licensing issues are gone for Mac (regarding FFmpeg and mp4’s).

As for the API simplifications, one sign it needed work was the growing number of capture methods. We’ve had captureMC, captureStage3D, iOS_captureFullscreen and recently also android_captureFullscreen. Another issue were some slight differences in how .init should be used, in regards to OpenGL capture on iOS(you had to set width & height to -1, very hacky), which resulted in if‘s to use different init configs depending on the platform. There were other issues with init as well (long param list even if not needed), and the names didn’t really express what the methods were doing as the meaning shifted for some platforms. For example,  encodeIt in the stone ages of FW really encoded all frames into the movie, but that is no longer true.

So here are the upcoming API changes you should prepare for:

  • init was renamed to start
  • in a typical case of fullscreen realtime capturing on any platform, you just call myEncoder.start(), with no arguments. This will automatically get stage dimensions for fullscreen capture.
  • if you need to capture downscaled video (typically in Flash to increase encoding speed), you specify your desired video dimensions similar to what you’ve done before and the captured frames will downscale automatically. You can use a new method myEncoder.setDimensions(w, h), or you could also set dimensions using myEncoder.start(fps, w, h, ….). In the past you had to use width / height and frameScale combo to achieve downscaling.
  • you can set your target fps either with myEncoder.setFps(fps), or inside start as mentioned above. If you don’t specify fps, it is automatically set to equal the fps of your Flash stage. An important new feature is, that in “realtime” mode frames are automatically skipped if needed to keep the video fps and stage fps in sync. For example, if your game is running at 60fps but your video target is 20fps, FlashyWrappers saves roughly every 3rd frame to compensate for the differences using some timer math. This worked only on iOS until now, while in 2.4+ it works on all platforms. So, you wont’t need to concern yourself about fps differences between video and Flash stage anymore.
  • all the XX_captureXX variants are being deprecated (they will still work but a compiler warning displays when using them) in favor of a single capture method. With no arguments, this method tries to capture fullscreen. Otherwise, you can give it DisplayObject or Context3D to capture. Hint: FW will try to accelerate fullscreen capture in realtime mode whenever possible, so use that for maximum speed (on iOS this takes advantage of OpenGL capturing).
  • formalized “realtime / non-realtime” modes – FW introduces a new switch in start called realtime. It’s a concept that FW has been working with all the time anyway(even mentioned in the docs), but until now it was up to the user to understand how to enable some kind of realtime capturing in the best way, usually by turning multithreading on (on some platforms), or by using iOS_captureFullscreen on iOS etc. From now on, this decision proccess is shifted towards FlashyWrappers. You just tell FW whenever you wish to capture in realtime or not and it switches multithreading or does other decisions which favor speed whenever needed. It also turns on/off fps syncing (frame skipping), which might not be desirable in non-realtime (post-processing) mode.
  • encodeIt renamed to finish.
  • all those changes and more which are not mentioned here are trying to make FlashyWrappers even simpler to use and more understandable & readable in the code. This is how exampleCamera FlashyWrappers core source code looks for all platforms in 2.4:


if (platform == 'FLASH') myEncoder.setDimensions(400, 300);
myEncoder.start();
...
myEncoder.capture();
...
myEncoder.finish();

Of course the events & saving video are left out, but it is hopefully clear how much simpler the code now looks. In 2.3, this is your exampleCamera init (now start) platform dependent code:

if (myEncoder.platform == "FLASH" || myEncoder.platform == "ANDROID") {
myEncoder.init(w, h, 20, 1000000, true, true, 44100, 1, 64000, 0.5);
} else {
if (myEncoder.platform == "IOS") {
myEncoder.init(-1, -1, 20, 1000000, true, true, 44100, 1, 64000);
} else {
myEncoder.init(w, h, 20, 1000000, true, true, 44100, 1, 64000);
}
}

 

Of course you’ll still be able to set some of the former parameters, but some of them are gone (like audio, frameScale, multithreading).

FW got new website / 3 amazing app videos / FW SDK 2.3

This post will be a bit more personal than usual, as this is another important milestone for FlashyWrappers.

If you want to skip to FW SDK 2.3 release info, scroll down

———————————————–

It’s finally here – everything FlashyWrappers related was moved to a new website at

http://flashywrappers.com/

In other words, FW has got its own website! This will allow for better expansion of the whole FlashyWrappers project and bringing the community of FW users together. There are always so many new ideas and features and fixes people need I doubt the project will stop anytime soon (despite what I thought in the beginning).

Not looking at the sales, the project always has been fun for me by offering sometimes very frustrating – yet rewarding – challenges, and that’s why I’m still working on it. The first 6 months working on and off on FlashyWrappers, I think I can reveal it made about 3 sales (it was the ancient version for $97, with 1 platform available). I can assure you, I would never continue working on it if it was just “business project”.

Since then the business situation has improved, luckily 🙂 But I think its clear that  there was always something drawing me back to this project, regardless of any sales or demand. It really was always more about those technical challenges. Can I capture AIR’s OpenGL on iOS, with 1 / 100 experience on iOS. Can I do the same on Android. And so on. The combined challenge of discovering new platforms, optimization techniques to beat the older and slower versions and really challenging myself combined with the visual reward – that is whats keeping me in.

Of course, I was hoping that some people would find this whole undertaking useful (I wouldn’t enjoy doing something people won’t use, in the long run). The first SWC I created was for my own video capturing project. And so when I found out much later (early this year) an app installed in Ray Ban booth (in Brazil!) will run on FlashyWrappers iOS, I was pretty excited.

FlashyWrappers inside
I mean really inside (on those iPads)

Anyway,  the new website contains a seed of showcase videos – the first 3 introducing Ray Ban app, ShadeScout app and Opera Maker app (all iOS). if you’ve got an app powered by FlashyWrappers, be sure to contact me through flashywrappers.com contact form! The new website will centralize everything FW related – I’m hoping to add forums for technical support later on (in case the user number grows even more).

This is a pretty niche market but I have the desire to improve FlashyWrappers step by step – still many challenges remain unsolved (if Keith is reading this, he knows… 🙂 and I think I won’t rest until I have the main priorities sorted out and people still need the library. There’s also room for expansion, the underlying libraries are native iOS / Android / Windows / Mac code so they don’t need to depend on AIR in the long run.

I’m happy the new website is finally up, it wasn’t as much work but the situation in my personal life is really crazy (in a bad way) so its good to see something coming together.

All the best with FlashyWrappers!

Pavel Langweil

———————————————–

FlashyWrappers SDK 2.3 was released, together with the new Android ANE preview! Grab it at the new website linked above.

FlashyWrappers 2.3
-------------------
- !! NEW PLATFORM PREVIEW ADDED: Android(HW) first look - Android ANE with mp4 hardware accelerated encoding and OpenGL fullscreen capture from AIR. This works only on Android 4.3+. No audio support / no captureMC support (audio support coming in next release). Do not use in production but let us know if it doesn't work on your device, with device log attached.
- Sound Mixer: FWSound mixer included into the package as it's now "SDK", so it should contain everything
- iOS: Can specify filename for saving the mp4 to the application storage. You will be responsible to erase these files if needed (instead of the default temp filename which is being reused).
- iOS: Fixed mp4 metatags from QuickTime to MPEG4 for better compatibility - it's possible to play the videos on Android now.
- iOS: Changed internal "hijacking" of AIR's OpenGL FBO - increases performance and reliability of the whole solution.
- iOS: Added support for recording from GPU mode in standard / high resolution in addition to Direct mode.
- iOS: Added "passthrough" encoding level for captureMC(keeps the movie exactly at the bitrate specified in init).

Solution for MP4’s desktop licensing & Android preview soon

The release of the Android HW encoder preview is almost ready (whats basically left is implementing  FW logo rendering in the demo version). Included in FlashyWrappers 2.3 will be also slightly updated iOS ANE. With that being said, here are some more details about what to expect and whats missing in the first “real” Android version.

Significantly increased speed – of course the biggest advantage of the new Android version is a major speed increase, which should allow realtime recording of AIR content in high resolution (webcam, games and so on). The tablet being used for all development tests has the following specs:

ASUS K01A – Intel Atom Z2520, CPU: 2×1,2 GHz, GPU: PowerVR SGX544MP2,

You can get one for < $100. In general, when not recording the fps stays at  constant 60fps, when recording 1024×768 @20 fps video from AIR, the apps fps fluctuates anywhere between 30-45, which leaves comfortable room to record 20fps videos. There is still room for improving the fps and it should get better in the future releases.

MP4 file output – by utilizing the device encoders you’ll get mp4 output natively, no ogv’s. This will alow saving to gallery later on(not yet implemented), or simply replaying the video on the phone, which was not possible with the ogv. There are no mp4 licensing issues (similar to iOS) thanks to using the device encoders.

NO audio support yet – the audio will be implemented in the next release. This release shouldn’t be used in production anyway – we’re curious on which devices the ANE “explodes”, as the video / GLES part is probably the most sensitive and device dependent part.

The release is probably 1-2 weeks away, depending on when the new FlashyWrappers website can be up and running.

And finally, what was that about desktop MP4’s? 

As you probably know, the whole MP4 business on desktop is one big mess currently, when it comes to licensing. Dealing wtih H.264 and AAC licensing +  OpenH264 isn’t exactly saving the day as that’s a partial solution just for the video (and it has its own gotchas). In Flash Player, that’s how it will stay for the time being unfortunately.

However, on AIR Mac and AIR Windows, change is coming. When using  encoders of the host operating system (OS X and Windows), the licensing is not an issue because similar to iOS and now Android, FlashyWrappers is utilizing the OS encoders, which are being licensed by Apple or Microsoft, not you.

On OS X the new Mac encoder is coming sooner than you think, as it’s quite simple to port the iOS encoder and utilize OS X AVFoundation H.264 and AAC encoders. In fact, we have a working Mac version right now and we’re testing it 🙂

Windows will need to be implemented from scratch, so the ETA on that is not known yet. Adding Windows platform and getting rid of ffmpeg in general will mean more overhead unfortunately, but we feel that getting rid of the licensing block is one of the most important thing we must do for FlashyWrappers to be really useful. And useful very much means supporting the most popular video format today (mp4), on all platforms with no strings attached (especially no licensing strings). Well, on all platforms except Flash. There’s no solution in sight there unless Adobe opens up their embedded H264 & AAC Flash encoder to developers. Because again, Adobe has licensed those encoders.