Recording Latency

edited August 2013

Hi Amazing Forum.

I've been trying this so hard with no success. We are playing a music, and recording audio on top of it. After that, we send that recording to the server and it will mix it for us with the Music.

The problem is, there is always a little latency added by the recording. I need to find what that latency is, and pass it along to the server. This is very normal behaviour and we have a flash version of the project that does the same thing.

But I can't seem to be able to get a call back when the recording started, and when the audio started. There seem to be only global Scheduler I can set, but those are not in sync with what I am trying to get. Or I just did't understand them correctly (Note that my ObjC is not that good)

I even tried to insert timestamps into AAE code directly, in output callbacks. But those timing are wrong.

Thank you for your help

Comments

  • So a bit of progress on this, I capture the time directly into renderCallback of the playing song, and audioCallback of the recording. Then I calculate the difference using CFAbsoluteTimeGetCurrent()

    But it is still 30ms off at times... I drag the result file and the song into audacity to compare the results. There is always something at the beginning of the recording that is not getting saved to the file. I am trying to know what that latency value is.

  • Well, it's working now. For those interested, I inserted callbacks in those two Audio Unit callbacks. My problem with the recording was I had to put the callback after
    if ( bufferLength > 0 ) {

    And I need to add 20ms to whatever result I get, because I am saving into m4a format and apparently that format does this.

  • @Daivuk

    Can you describe exactly what you did, as I'm struggling to resolve a record-while-play latency issue as well, and need to be able to correct for this.

    @Michael - any approved methods for this?

    Many thanks

    Mike

  • Follow-up, having put a Log to record the input and output latency at record time it looks like I'm getting 0.01 latency (which I assume is in seconds, so 10ms) in each step.

    I probably therefore need to trim 0.02s from the track before I save it - thoughts on the best way to do this?

  • The best - and built-in - way to handle latency is to use the audio timestamps. These correspond to the moment audio will be heard (for output audio), or the moment the audio hit the microphone (for recorded audio). *

    AERecorder itself actually already does all the latency adjustment for you (via AEMixerBuffer) if you're playing and recording on the same device - you just need to pass it the correct timestamps from the various callbacks.


    * One caveat - for whatever reason, this isn't completely true, as you need to add/subtract inputLatency/outputLatency to get the true value. Don't look at me, it's Core Audio =)

  • @Michael,

    So should aerecorder be creating latency free content? As currently there's definitely a noticeable 20ms lag if I record and play-record

  • Hadn't actually looked into AE mixer buffer yet - is there a sample code for passing the aemixerbuffer time codes that I've somehow missed?

  • That's right - put more clearly, it should be creating synchronised content, so that the effects of latency can't be heard on playback. If that's not happening, something else is going on, or I've buggered something up =) The sample app plays and records, if I remember correctly - does that work right?

  • This is a super old thread, but I thought I'd share my experience with this. When recording to m4a format with AERecorder, I have found that the first 50-200ms of sound processed by AERecorder is simply not saved to the file. You can modify AERecorder, or create your own audio receiver that works in parallel, to count total frames received by the audioCallback. That number of frames will always be more than the number of frames in the file, and it is because the first chunk of sound simply isn't there.

    The frame difference varies by device, and I don't know any realtime way to detect what that difference is.

  • I think this issue is due to the AEMixerBufferEnqueue being called the first time. Michael has said in the docs that the mixer may discard a buffer or two while it starts up the first time. The solution is to call:

    AEMixerBufferEnqueue(mixer, source, NULL, 0, NULL);

    prior to the actual recording taking place. It will then record all the data sent it's way.

    That said, there is still some unknown causes of latency that I'm seeing, especially on a iphone6+.

  • edited October 2015

    @Michael said:
    The best - and built-in - way to handle latency is to use the audio timestamps. These correspond to the moment audio will be heard (for output audio), or the moment the audio hit the microphone (for recorded audio). *

    AERecorder itself actually already does all the latency adjustment for you (via AEMixerBuffer) if you're playing and recording on the same device - you just need to pass it the correct timestamps from the various callbacks.


    * One caveat - for whatever reason, this isn't completely true, as you need to add/subtract inputLatency/outputLatency to get the true value. Don't look at me, it's Core Audio =)

    This doesn't seem to be true. If I playback a file, and record an m4a file of the voice, I cannot get them to sync. When I get the audio timestamps from the file player and aerecorder, they don't correspond to the actual delay between the original playback file and the recorded file.

    A test is to playback music, and record the audio output over the mic and manually determine the sync value in Audacity. If I call the recorder, then the playback, the latency should be the difference between the two. Sometimes it is, but often its not. Sometimes its sample accurate, sometimes it's off by anything up to 40ms either way. :(

  • Interesting! Can you reproduce those results with the sample app, with your setup? (turn on, say, the drums, then turn up the input gain, hit "record", and take a look at the recorded audio. If you slide the input gain up and down it makes it easier to hear the difference between the outgoing and outgoing+incoming audio).

  • Another note: in the in-development version, AEAudioController's automaticLatencyManagement is now on by default, but if you're using an older version, be sure to enable it (just set the property to YES on startup).

  • Hi Michael. I think I'm going to build a little test app if I can't get this straight, and I'll throw it up on github for you to look at. I have one more idea to try out first and I'll report back.

Sign In or Register to comment.