| Summary: | SDL_GetQueuedAudioSize is broken with WASAPI | ||
|---|---|---|---|
| Product: | SDL | Reporter: | Giovanni Bajo <rasky> |
| Component: | audio | Assignee: | Ryan C. Gordon <icculus> |
| Status: | RESOLVED FIXED | QA Contact: | Sam Lantinga <slouken> |
| Severity: | blocker | ||
| Priority: | P2 | CC: | bibendovsky, cameron.gutman, sezeroz |
| Version: | 2.0.8 | ||
| Hardware: | x86 | ||
| OS: | Windows 7 | ||
| Attachments: |
Test program showing the bug
Another sample program without input data A patch for the bug |
||
Strictly speaking, this isn't a bug, since this is the number of bytes of queued data the system reports, but we might be able to estimate this better ourselves. I'll take a look when I get a moment. --ryan. (In reply to Ryan C. Gordon from comment #1) > Strictly speaking, this isn't a bug, since this is the number of bytes of > queued data the system reports, but we might be able to estimate this better > ourselves. > > I'll take a look when I get a moment. It looks like there's a fixed offset that's always added, but it changes between executions of the same program. In the first WASAPI example, all reported numbers have an additional offset of 7056: the playback starts at 65536+7056=72592, and stops at 0+7056=7056. In the second example, the offset is 3528. (In reply to Ryan C. Gordon from comment #1) > Strictly speaking, this isn't a bug, since this is the number of bytes of > queued data the system reports, but we might be able to estimate this better > ourselves. > > I'll take a look when I get a moment. > > --ryan. Encountered the issue too. The function returns non-zero value even if you did not queue any data at all. Just after unpausing an audio the function returns zero, but shortly after it begins to return some non-zero values. Created attachment 3292 [details]
Another sample program without input data
A sample program for testcase.
Does not queue data at all, just polling for queued data size.
Should output non-zero values on WASAPI audio driver.
Created attachment 3293 [details]
A patch for the bug
Removes GetPendingBytes since it does not make sense here.
Ryan, can you look at this for SDL 2.0.9? I too encountered this issue and came to the same conclusion. I was trying to use SDL_GetQueuedAudioSize() to ensure my audio latency didn't get too high while streaming data in from the network. If I get more than N frames of audio queued, I know that the network is giving me more data than I can play and I need to drop some to keep latency low. This doesn't work well on WASAPI out of the box, due to the addition of GetPendingBytes() to the amount of queued data. As a terrible hack, I loop 100 times calling SDL_Delay(10) and SDL_GetQueuedAudioSize() before I ever call SDL_QueueAudio() to get a "baseline" amount that I then subtract from SDL_GetQueuedAudioSize() later. However, because this value isn't actually a constant, this hack can cause SDL_GetQueuedAudioSize() - baselineSize to be < 0. This means I have no accurate way of determining how much data is actually queued in SDL's audio buffer queue. The SDL_GetQueuedAudioSize() documentation says: "This is the number of bytes that have been queued for playback with SDL_QueueAudio(), but have not yet been sent to the hardware." Yet, SDL_GetQueuedAudioSize() returns > 0 value when SDL_QueueAudio() has never been called. Based on that documentation, I believe the current behavior contradicts the documented behavior of this function and should be changed in line with Boris's patch. I understand that exposing the IAudioClient::GetCurrentPadding() value is useful, but a solution there needs to take into account what of that data is silence inserted by SDL and what is actual data queued by the user with SDL_QueueAudio(). Until that happens, I think the best approach is to remove the GetPendingBytes() call until SDL is able to keep track of queued data to make sense of it. This would make SDL_GetQueuedAudioSize() possible to use accurately with WASAPI. Agreed. Ryan, can you look at this along with your other WASAPI work for 2.0.9? (In reply to Sam Lantinga from comment #8) > Agreed. Ryan, can you look at this along with your other WASAPI work for > 2.0.9? Yep! --ryan. Just to follow up, this is messier than it initially looks, because it doesn't take into account conversion streams and such (which WASAPI is more likely to use than DirectSound), so this is going to wait until after 2.0.9. --ryan. (In reply to Ryan C. Gordon from comment #10) > Just to follow up, this is messier than it initially looks, because it > doesn't take into account conversion streams and such (which WASAPI is more > likely to use than DirectSound), so this is going to wait until after 2.0.9. > > --ryan. Why it should take into account conversion streams in first place? Audio queue (SDL_dataqueue.c) by itself does not make any conversions at all. User manages that queue via appropriate API functions: SDL_ClearQueuedAudio, SDL_DequeueAudio, SDL_QueueAudio, SDL_GetQueuedAudioSize. On the contrary, GetPendingBytes function related to device's audio queue not the user's one. Also, audio thread keeps filling this queue with silence even the user does not queue any data at all. The point is: SDL_GetQueuedAudioSize must return the size of user's audio queue only. By the way, WASAPI backend is the only one which implements GetPendingBytes. The rest backends uses default implementation which returns zero by default.
> The point is: SDL_GetQueuedAudioSize must return the size of user's audio
> queue only.
No, it returns the size of the user's audio queue, counting data that has been handed to the hardware but not yet played.
The problem is that there is data that leaves the user's audio queue and enters a different queue that is converting between the format the user is feeding in and the format the hardware wants, and SDL_GetQueuedAudioSize currently ignores that. Most backends do their own conversion and resampling and thus never use this second queue; WASAPI never does this for us.
The GetPendingBytes interface was probably a mistake, though...it was added at the request of a specific developer that wanted the absolute lowest latency from SDL_QueueAudio(). I'm pretty sure that after demanding this, he never used SDL anyhow, though.
Sam: should I just nuke this interface and treat audio as "consumed" once SDL pulls it out of the queue at all?
--ryan.
Any plans to fix it in 2.0.10? Yes, Ryan, that sounds like a good idea. It'll be in line with the other audio drivers and less confusing for API consumers. This is fixed, thanks! https://hg.libsdl.org/SDL/rev/04552ff8c83d |
Created attachment 3247 [details] Test program showing the bug SDL_GetQueuedAudioSize should report the number of queued bytes that have not yet been played. I don't need this number to be very precise, but if I queue some audio and wait, I expect the reported number to be monotonically decreasing, and eventually reach zero to report that all queued audio was played. The simple example attached to this bug shows that this expectation is correct with both the DirectSound and winmm backend, but not when using WASAPI. This is the output with winmm: We compiled against SDL version 2.0.8 ... But we are linking against SDL version 2.0.8. INFO: Using audio driver: winmm Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 65536 65536 57344 49152 49152 49152 40960 32768 32768 24576 16384 8192 8192 8192 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 This is the output with DirectSound: We compiled against SDL version 2.0.8 ... But we are linking against SDL version 2.0.8. INFO: Using audio driver: directsound Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 65536 65536 65536 57344 49152 49152 40960 40960 32768 24576 24576 16384 16384 8192 8192 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 This is the output using WASAPI: We compiled against SDL version 2.0.8 ... But we are linking against SDL version 2.0.8. INFO: Using audio driver: wasapi Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 72592 72592 56208 39824 39824 23440 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 7056 Notice that the reported number at the end of playback is also not very stable; sometimes the number is different: We compiled against SDL version 2.0.8 ... But we are linking against SDL version 2.0.8. INFO: Using audio driver: wasapi Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 Initial buffer: 0 69064 52680 36296 19912 19912 3528 3528 7056 7056 7056 7056 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 3528 NOTE: I'm testing it on Windows 7 running within Parallels under macOS. I don't think that matters much, but I wanted to give complete information.