Tutorials, Videos

AIR, Android and the microphone

21 Comments 19 August 2010

AIR, Android and the microphone

After having played with my phone’s camera in a previous post, I’d like to share with you a new experiment based on the microphone. My teammates already posted amazing applications using Adobe AIR 2.5 on Android and the new microphone API. My favorite one has been developed by Christophe Coenraets: “VoiceNotes for Android”. A video and the source code of this Flex application are available on his blog. The only issue with the app is that it stores raw data of the sound recorded by the microphone. So here is the result of my experiment: my AIR app for Android records your voice, then it produces a WAV file, encodes it to MP3 and saves it on your SDcard.

To create a WAV file I’m using the WAVwriter.as class available here: http://code.google.com/p/ghostcat/source/browse/trunk/ghostcatfp10/src/ghostcat/media/WAVWriter.as?spec=svn424&r=424

To encode the MP3 file, I’m using Shine, an AS3 library that leverages the ability of Alchemy to run C libraries (LAME). It’s quite slow on my phone, but at least I can save MP3 files directly on my SDcard. Here is the link to the official page of the Shine project: http://code.google.com/p/flash-kikko/

I also used the “Nintendo DS trick” for gaming interactions. If you blow on your phone, then you’ll see Mary Poppins fly. Using the accelerometer, you can make her move from left to right on your screen. I’ve already detailed how to use the accelerometer in this article: http://www.riagora.com/2010/04/air-and-the-accelerometer/

Here is a video that presents the application:

Android 2.2, AIR 2.5 and the microphone from michael chaize on Vimeo.

Here is the source code of this application:

<?xml version='1.0' encoding='UTF-8'?>
<s:Application    xmlns:d="http://ns.adobe.com/fxg/2008/dt" creationComplete="application1_creationCompleteHandler(event)" xmlns:fx="http://ns.adobe.com/mxml/2009" xmlns:s="library://ns.adobe.com/flex/spark" width="480" height="800" backgroundColor="#FFFFFF" preloaderChromeColor="#FFFFFF" xmlns:components="components.*" xmlns:local="*" >
	<fx:Style source="Main.css"/>
 
	<fx:Script>
		<![CDATA[
			import com.adobe.audio.format.WAVWriter;
 
			import flash.events.SampleDataEvent;
			import flash.media.Microphone;
			import flash.media.Sound;
			import flash.sensors.Accelerometer;
			import flash.utils.ByteArray;
 
			import fr.kikko.lab.ShineMP3Encoder;
 
			import mx.events.FlexEvent;
 
			//variables to move Mary Poppins with the accelerometer
			private var accX:Number = 0;
			private var myAcc:Accelerometer = new Accelerometer();
			private var vAcceleration:Number = 0.5;
			private var vVelocity:Number = -10;
 
			//variables to play with the microphone
			private  var microphone:Microphone;
			private var soundRecording:ByteArray;
			private var soundOutput:Sound;
			private var mp3encoder:ShineMP3Encoder;
 
			private function startMicRecording():void
			{
				soundRecording = new ByteArray();
				microphone.addEventListener(SampleDataEvent.SAMPLE_DATA, gotMicData);
				//UI
				btnRecord.enabled = false;
				btnStop.enabled = true;
				btnPlayback.enabled = false;
				myTI.text = "Press stop";
			}
 
			private function stopMicRecording():void
			{
				microphone.removeEventListener(SampleDataEvent.SAMPLE_DATA, gotMicData);
				//UI
				soundRecording.position = 0;
				btnRecord.enabled = true;
				btnStop.enabled = false;
				btnPlayback.enabled = true;
				convert2MP3();
				myAiguille.rotation = -40;
			}
 
			private function gotMicData(micData:SampleDataEvent):void
			{
				//This is currently recording my voice
				soundRecording.writeBytes(micData.data);
			}
 
			private function getMicActivity(micData:SampleDataEvent):void{
				//To update the Vu-meter indicator
				myAiguille.rotation = 8/10 * microphone.activityLevel - 40;
			}
 
			private function playbackData():void
			{
				//Playback the raw sound data stored in memory
				soundRecording.position = 0;
				soundOutput = new Sound();
				soundOutput.addEventListener(SampleDataEvent.SAMPLE_DATA, playSound);
				soundOutput.play();
			}
 
			private function playSound(soundOutput:SampleDataEvent):void
			{
				trace("inPlaySound");
				if (!soundRecording.bytesAvailable > 0)
					return;
				for (var i:int = 0; i < 8192; i++)
				{
					var sample:Number = 0;
					if (soundRecording.bytesAvailable > 0)
						sample = soundRecording.readFloat();
					soundOutput.data.writeFloat(sample);
					soundOutput.data.writeFloat(sample);
				}
			}
 
			private function convert2MP3():void{
				//First you need to create a WAV file using WAVWriter
				//Then I can encode the WAV file into a MP3 file using Shine
				var wavWrite:WAVWriter = new WAVWriter();
				wavWrite.numOfChannels = 1;
				wavWrite.sampleBitRate = 16;
				wavWrite.samplingRate = 44100;
 
				var wav:ByteArray = new ByteArray();
 
				wavWrite.processSamples(wav, soundRecording, 44100,1);
				wav.position = 0;
 
				mp3encoder = new ShineMP3Encoder(wav);
 
				mp3encoder.addEventListener(Event.COMPLETE, onEncoded);
				mp3encoder.addEventListener(ProgressEvent.PROGRESS, onEncodingProgress);
				myTI.text = "Encoding MP3 file...";
				mp3encoder.start();
			}
 
			private function onEncoded(e:Event):void{
				myTI.text = "Mp3 encoded and saved on your SDcard. Press Play.";
				mp3encoder.mp3Data.position = 0;
				var myDate:Date = new Date();
				var theDate:String = myDate.monthUTC.toString()+myDate.dayUTC.toString()+myDate.hoursUTC.toString()+myDate.minutesUTC.toString()+myDate.secondsUTC.toString();
				mp3encoder.saveAs("myVoice"+theDate+".mp3");
			}
 
			private function onEncodingProgress(evt:ProgressEvent):void{
				myTI.text = "Encoding MP3 "+ evt.bytesLoaded.toString() + "/" + evt.bytesTotal.toString();
			}
 
			protected function application1_creationCompleteHandler(event:FlexEvent):void
			{
				//Starts the microphone to update the UI
				microphone = Microphone.getMicrophone(-1);
				microphone.rate = 44;
				mary.addEventListener(Event.ENTER_FRAME, onEnterMary);
				myAcc.addEventListener(AccelerometerEvent.UPDATE, onAccUpdate);
				microphone.addEventListener(SampleDataEvent.SAMPLE_DATA, getMicActivity);
			}
 
			private function onAccUpdate(evt:AccelerometerEvent):void{
				//Mary Poppins moves from left to right with the accelerometer
				accX = evt.accelerationX * (-1);
			}
 
			private function onEnterMary(evt:Event):void{
				//Logic to make Mary fly
 
				mary.y += vVelocity;
				vVelocity += vAcceleration;
 
				if (mary.y > 635) {
					vVelocity = 0;
					mary.y = 635;
				}else{
					// the treshold is 75 (when you blow on a microphone)
					if(microphone.activityLevel > 75){
						vVelocity = -20;
					}
 
					mary.x -= (mary.x - (mary.x + accX * 10))*0.6;
				}
				if (mary.x > 480) mary.x = 0;
				if (mary.x < 0) mary.x = 480;
			}
 
		]]>
	</fx:Script>
	<!--Code generated by Flash Catalyst CS5-->
	<s:BitmapImage smooth="true" source="@Embed('/assets/images/microphoneAndroid/Background.png')" d:userLabel="Background" x="0" y="0"/>
	<s:BitmapImage smooth="true" source="@Embed('/assets/images/microphoneAndroid/Layer 1.png')" d:userLabel="Layer 1" x="0" y="30"/>
	<s:BitmapImage smooth="true" source="@Embed('/assets/images/microphoneAndroid/Layer 10.png')" d:userLabel="Layer 10" x="0" y="191"/>
	<s:Button click="startMicRecording()"  skinClass="components.Layer8Button" x="36" y="86" id="btnRecord"/>
	<s:Button click="playbackData()" skinClass="components.Layer8copyButton2" x="192" y="86" id="btnPlayback" enabled="false"/>
	<s:BitmapImage alpha="0.07" smooth="true" source="@Embed('/assets/images/microphoneAndroid/Layer 6.png')" d:userLabel="Layer 6" x="0" y="30"/>
	<s:BitmapImage blendMode="multiply" smooth="true" source="@Embed('/assets/images/microphoneAndroid/Layer 7_s Drop Shadow.png')" d:userLabel="Layer 7's Drop Shadow" x="-4" y="-4"/>
	<s:BitmapImage smooth="true" source="@Embed('/assets/images/microphoneAndroid/Layer 7.png')" d:userLabel="Layer 7" x="0" y="0"/>
	<s:RichText color="#ffffff" fontFamily="Adobe Clean" fontSize="14" kerning="off" lineHeight="120%" d:userLabel="ADOBE AIR AND THE MICROPHONE" whiteSpaceCollapse="preserve" x="10" y="11">
		<s:content><s:p><s:span>ADOBE AIR AND THE MICROPHONE</s:span></s:p></s:content>
	</s:RichText>
	<s:Button click="stopMicRecording()" skinClass="components.Layer9Button" x="110" y="87" id="btnStop" enabled="false"/>
	<s:BitmapImage smooth="true" source="@Embed('/assets/images/microphoneAndroid/Layer 2.png')" d:userLabel="Layer 2" visible="false" x="266" y="77"/>
	<s:BitmapImage smooth="true" source="@Embed('/assets/images/microphoneAndroid/Layer 12.png')" d:userLabel="Layer 12" x="7" y="218"/>
	<components:Layer11CustomComponent x="179" y="635" id="mary" />
	<s:Label x="10" y="198" text="Label" width="460" color="#EEEEEE" fontFamily="Verdana" fontSize="10" id="myTI"/>
	<local:aiguille id="myAiguille" x="398" y="155" height="90" width="2" rotation="-39"/>
</s:Application>

Here is a link to the apk file (you need to install the Adobe AIR 2.5 runtime to make it work – version from the 8th of August on labs.adobe.com): http://riagora.com/pvt_content/android/AIR_microphone.apk

And finally a link to the Flash Builder project:
http://riagora.com/pvt_content/android/AIR_microphone.zip

Enjoy.

Post to Twitter

Your Comments

21 Comments so far

  1. Francisc says:

    Excellent article, Michaël. You’re one of the best Flex Evangelists we have.

  2. admin says:

    Well… thanks. I think I’ll write an article on the Evangelism team sooner or later, to introduce all my teammates.

  3. Ecevit says:

    Hi Michael, one big question for you. Yesterday Apple changed a major restriction concerning 3rd part apps generating iOS application. It appears that Flash or Air generated application will no longer be excluded from appstore. Do you know if Adobe will re-start working on their product to fully support iOS application generation?

    PS : nice blog by the way

  4. admin says:

    Hi. I’m still waiting for the official statement regarding the iPhone packager. Anyway… I so happy for Flash developers. Now we can target Android phones and tablets, Apple phones and tablets… and soon Blackberries, etc…

  5. Jack says:

    I’m trying to get started and am lost in flex vs AS vs AIR vs … My goal is to get this working on an android device.

    However, when I load the code into flex burrito, it is designed to run on a desktop not a device. Then when I try to create a mobile app and load the code in that, I get issues on the combobox, button and textInput objects.

    I’d love to get this or (preferably) the voicenotes app working. I’m working in WIN with burrito but can change to anything other than MAC based. I just need to get started with an audio play/record type app.

  6. admin says:

    Hi Jack,
    I’ll try to update my app with the new Flex Mobile framework ‘Hero’.
    It’s using a new set of components, but you can import the classic swc files to make it work. When you create a new mobile project, you can choose to create either a mobile application, or a blank application. Choose the second one. But still… I’ll try to update my application to run it using the classic Burrito configuration.

  7. Jack says:

    Hi,

    Thanks for the reply. Unfortunately for someone completely new to the tools and Adobe process “importing the classic swc files” doesnt mean very much. Can you give more detailed directions? Where do I get these swv files and where do I put them?

    Or, for that matter, what is an swv file?

  8. Great stuff. It looks like the user can change the name of the file when saving and choose whatever location they would like. This seems to present a problem in my use case where I simply want to save in a location of my choosing (I was storing the wav files in the documents directory). I was fine with this but ran into trouble with the wav playback :(

    Is there any way to save the MP3 without the prompt?

  9. Cboudreau says:

    Nice explanation on how to use the microphone on an Android phone!

    Thank you.

  10. FTQuest says:

    Hello Michael,

    Could you explain to me (or point me to the sources that would explain) the need to have the raw data from microphone being first formatted as wav and only then encoded as mp3?
    In other words, what is the need to have an intermediate stage?

    thanks,
    Igor

  11. admin says:

    It must be due to a transcription from a C library to a AS3 library using the Alchemy project. I guess that the original C library is expecting a well formatted WAV file that’s it. I couldn’t find a pure AS3 library able to compress the raw data directly into MP3.

  12. Drew says:

    Hi Michael,

    Thanks for the great tutorial.
    In your Flash Builder Project .zip, the asset folder (/assets/images/microphoneAndroid/) isn’t there. Was it supposed to be packaged in there?
    Thanks,

    ~D

  13. Saar says:

    has anyone succeeded in recording mic audio and playing back on ios?
    i get a lot of problems!

    (works great on desktop and android)

  14. Tim says:

    Hi – I am new to Flex (quite experienced with Flash) – loving your work! Is there an example of how you did the needle rotation. I could not find it in the project zip. Thanks, Tim

  15. Ed says:

    Can this be used in Flash cs5.5?


Trackbacks/Pingbacks

  1. AIR 2.5 et Android: les news « Code moi un mouton - August 23, 2010
  2. AIR, Android and the microphone | Android Air Apps - October 4, 2010

    [...] Read the full article and get the source code here. [...]

  3. AIR 2.5 pour Android est live « Code moi un mouton - October 12, 2010

    [...] – Comment jouer avec le microphone: http://www.riagora.com/2010/08/air-android-and-the-microphone/ [...]

  4. AIR, Android and the microphone | RIAgora | marcusjpotter - January 5, 2011

    [...] AIR, Android and the microphone | RIAgora. //Still seems one of the few workarounds to to save WAV or MP3 to SD card with AIR Android platform… Guessing, only other solution would to experiment on integrating Java with AIR (although this area is unexplored) [...]

  5. Quiet outdoor condenser - - Mini Split Air - August 22, 2011

    [...] AIR, Android and the microphone | RIAgora [...]

  6. Air 2.5 et stratégie multi-terminal | Blog MTI - June 17, 2012

    [...] Android Microphone [...]

Share your view

Post a comment

Who am I ?

I'm Michaël CHAIZE, Adobe Flash Platform Evangelist based in Paris. I'm a big fan of Rich Internet Applications and I promote the Flash Platform in the Enterprise world.
You can follow me on twitter: http://twitter.com/mchaize

Magazine

Follow us on Facebook

© 2014 RIAgora. Powered by WordPress.

Daily Edition Theme by WooThemes - Premium WordPress Themes