Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDL_GetQueuedAudioSize is broken with WASAPI #2902

Closed
SDLBugzilla opened this issue Feb 11, 2021 · 0 comments
Closed

SDL_GetQueuedAudioSize is broken with WASAPI #2902

SDLBugzilla opened this issue Feb 11, 2021 · 0 comments

Comments

@SDLBugzilla
Copy link
Collaborator

This bug report was migrated from our old Bugzilla tracker.

These attachments are available in the static archive:

Reported in version: 2.0.8
Reported for operating system, platform: Windows 7, x86

Comments on the original bug report:

On 2018-05-19 09:03:46 +0000, Giovanni Bajo wrote:

Created attachment 3247
Test program showing the bug

SDL_GetQueuedAudioSize should report the number of queued bytes that have not yet been played. I don't need this number to be very precise, but if I queue some audio and wait, I expect the reported number to be monotonically decreasing, and eventually reach zero to report that all queued audio was played.

The simple example attached to this bug shows that this expectation is correct with both the DirectSound and winmm backend, but not when using WASAPI.

This is the output with winmm:

We compiled against SDL version 2.0.8 ...
But we are linking against SDL version 2.0.8.
INFO: Using audio driver: winmm
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
65536
65536
57344
49152
49152
49152
40960
32768
32768
24576
16384
8192
8192
8192
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

This is the output with DirectSound:

We compiled against SDL version 2.0.8 ...
But we are linking against SDL version 2.0.8.
INFO: Using audio driver: directsound
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
65536
65536
65536
57344
49152
49152
40960
40960
32768
24576
24576
16384
16384
8192
8192
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

This is the output using WASAPI:

We compiled against SDL version 2.0.8 ...
But we are linking against SDL version 2.0.8.
INFO: Using audio driver: wasapi
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
72592
72592
56208
39824
39824
23440
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056
7056

Notice that the reported number at the end of playback is also not very stable; sometimes the number is different:

We compiled against SDL version 2.0.8 ...
But we are linking against SDL version 2.0.8.
INFO: Using audio driver: wasapi
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
Initial buffer: 0
69064
52680
36296
19912
19912
3528
3528
7056
7056
7056
7056
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528
3528

NOTE: I'm testing it on Windows 7 running within Parallels under macOS. I don't think that matters much, but I wanted to give complete information.

On 2018-05-19 17:46:23 +0000, Ryan C. Gordon wrote:

Strictly speaking, this isn't a bug, since this is the number of bytes of queued data the system reports, but we might be able to estimate this better ourselves.

I'll take a look when I get a moment.

--ryan.

On 2018-05-20 12:51:03 +0000, Giovanni Bajo wrote:

(In reply to Ryan C. Gordon from comment # 1)

Strictly speaking, this isn't a bug, since this is the number of bytes of
queued data the system reports, but we might be able to estimate this better
ourselves.

I'll take a look when I get a moment.

It looks like there's a fixed offset that's always added, but it changes between executions of the same program. In the first WASAPI example, all reported numbers have an additional offset of 7056: the playback starts at 65536+7056=72592, and stops at 0+7056=7056. In the second example, the offset is 3528.

On 2018-05-22 17:59:35 +0000, Boris Bendovsky wrote:

(In reply to Ryan C. Gordon from comment # 1)

Strictly speaking, this isn't a bug, since this is the number of bytes of
queued data the system reports, but we might be able to estimate this better
ourselves.

I'll take a look when I get a moment.

--ryan.

Encountered the issue too.

The function returns non-zero value even if you did not queue any data at all.

Just after unpausing an audio the function returns zero, but shortly after it begins to return some non-zero values.

On 2018-09-02 09:56:59 +0000, Boris Bendovsky wrote:

Created attachment 3292
Another sample program without input data

A sample program for testcase.
Does not queue data at all, just polling for queued data size.
Should output non-zero values on WASAPI audio driver.

On 2018-09-02 10:03:51 +0000, Boris Bendovsky wrote:

Created attachment 3293
A patch for the bug

Removes GetPendingBytes since it does not make sense here.

On 2018-10-01 17:00:10 +0000, Sam Lantinga wrote:

Ryan, can you look at this for SDL 2.0.9?

On 2018-10-10 07:59:19 +0000, Cameron Gutman wrote:

I too encountered this issue and came to the same conclusion. I was trying to use SDL_GetQueuedAudioSize() to ensure my audio latency didn't get too high while streaming data in from the network. If I get more than N frames of audio queued, I know that the network is giving me more data than I can play and I need to drop some to keep latency low.

This doesn't work well on WASAPI out of the box, due to the addition of GetPendingBytes() to the amount of queued data. As a terrible hack, I loop 100 times calling SDL_Delay(10) and SDL_GetQueuedAudioSize() before I ever call SDL_QueueAudio() to get a "baseline" amount that I then subtract from SDL_GetQueuedAudioSize() later. However, because this value isn't actually a constant, this hack can cause SDL_GetQueuedAudioSize() - baselineSize to be < 0. This means I have no accurate way of determining how much data is actually queued in SDL's audio buffer queue.

The SDL_GetQueuedAudioSize() documentation says: "This is the number of bytes that have been queued for playback with SDL_QueueAudio(), but have not yet been sent to the hardware." Yet, SDL_GetQueuedAudioSize() returns > 0 value when SDL_QueueAudio() has never been called.

Based on that documentation, I believe the current behavior contradicts the documented behavior of this function and should be changed in line with Boris's patch.

I understand that exposing the IAudioClient::GetCurrentPadding() value is useful, but a solution there needs to take into account what of that data is silence inserted by SDL and what is actual data queued by the user with SDL_QueueAudio(). Until that happens, I think the best approach is to remove the GetPendingBytes() call until SDL is able to keep track of queued data to make sense of it. This would make SDL_GetQueuedAudioSize() possible to use accurately with WASAPI.

On 2018-10-10 15:51:17 +0000, Sam Lantinga wrote:

Agreed. Ryan, can you look at this along with your other WASAPI work for 2.0.9?

On 2018-10-10 17:57:18 +0000, Ryan C. Gordon wrote:

(In reply to Sam Lantinga from comment # 8)

Agreed. Ryan, can you look at this along with your other WASAPI work for
2.0.9?

Yep!

--ryan.

On 2018-10-30 03:26:12 +0000, Ryan C. Gordon wrote:

Just to follow up, this is messier than it initially looks, because it doesn't take into account conversion streams and such (which WASAPI is more likely to use than DirectSound), so this is going to wait until after 2.0.9.

--ryan.

On 2018-10-31 07:59:49 +0000, Boris Bendovsky wrote:

(In reply to Ryan C. Gordon from comment # 10)

Just to follow up, this is messier than it initially looks, because it
doesn't take into account conversion streams and such (which WASAPI is more
likely to use than DirectSound), so this is going to wait until after 2.0.9.

--ryan.

Why it should take into account conversion streams in first place?

Audio queue (SDL_dataqueue.c) by itself does not make any conversions at all.

User manages that queue via appropriate API functions: SDL_ClearQueuedAudio, SDL_DequeueAudio, SDL_QueueAudio, SDL_GetQueuedAudioSize.

On the contrary, GetPendingBytes function related to device's audio queue not the user's one. Also, audio thread keeps filling this queue with silence even the user does not queue any data at all.

The point is: SDL_GetQueuedAudioSize must return the size of user's audio queue only.

By the way, WASAPI backend is the only one which implements GetPendingBytes. The rest backends uses default implementation which returns zero by default.

On 2018-11-22 06:57:36 +0000, Ryan C. Gordon wrote:

The point is: SDL_GetQueuedAudioSize must return the size of user's audio
queue only.

No, it returns the size of the user's audio queue, counting data that has been handed to the hardware but not yet played.

The problem is that there is data that leaves the user's audio queue and enters a different queue that is converting between the format the user is feeding in and the format the hardware wants, and SDL_GetQueuedAudioSize currently ignores that. Most backends do their own conversion and resampling and thus never use this second queue; WASAPI never does this for us.

The GetPendingBytes interface was probably a mistake, though...it was added at the request of a specific developer that wanted the absolute lowest latency from SDL_QueueAudio(). I'm pretty sure that after demanding this, he never used SDL anyhow, though.

Sam: should I just nuke this interface and treat audio as "consumed" once SDL pulls it out of the queue at all?

--ryan.

On 2019-06-04 10:41:57 +0000, Boris Bendovsky wrote:

Any plans to fix it in 2.0.10?

On 2019-06-05 00:22:59 +0000, Sam Lantinga wrote:

Yes, Ryan, that sounds like a good idea. It'll be in line with the other audio drivers and less confusing for API consumers.

On 2019-06-05 00:32:36 +0000, Sam Lantinga wrote:

This is fixed, thanks!
https://hg.libsdl.org/SDL/rev/04552ff8c83d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant