1454059 - Running a video claims 1-2 MiB/s into heap-unclassified without ever releasing it

Status: UNCONFIRMED

(Core :: Audio/Video: Playback, defect)

Description

[sworddragon2]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Build ID: 20100101

Steps to reproduce:

1. Start any stream on Twitch.
2. Watch the related content-process for some time with a process-viewer.
3. Close the tab containing the Twitch stream.


Actual results:

1. During video playback the memory usage grows about 1-2 MiB/s (it stops if the video is paused).
2. The content process has nearly an idle CPU usage (~1%) while the main process increases significantly with CPU usage.
3. Closing the tab does not close the related content process or releases its potential huge amount of claimed memory.


Expected results:

Unsure what is wrong at all but at least the infinite memory claiming should not happen.


Additional information:

I noticed this because a firefox content process had claimed over 12 GiB into the resident size set and allocated a total of over 19 GiB of memory (with over 98% being listed under heap-unclassified on about:memory). The system froze for a few seconds (eventually from late swapping on a HDD) and I heard a loud crackling sound and the video stopped while the pulseaudio process drove crazy with 100% CPU usage on 1 core (it required a SIGKILL to be stopped and possibly the Firefox behavior just revealed a pulseaudio bug here too).


This issue reproduces always here with Firefox 59.0.2 on Linux and since I'm looking not that rarely videos on Twitch for a long time this issue seems to be a not so old regression.

Comment 1

[ovidiu boca[:Ovidiu]]

Hi sworddragon2,

I tested this issue on Ubuntu 16.04 with FF 59 release and FF Nightly 61.0a1(2018-04-17) and I can't reproduce this issue. 

You can try to see if this issue is reproducible in Safe Mode, here is a link that can help you:
https://support.mozilla.org/en-US/kb/troubleshoot-firefox-issues-using-safe-mode

If the issue is repro in safe mode please install FF Nightly 61.0a1 and see if the memory usage grows. Download Firefox Nightly from here: https://nightly.mozilla.org/

One more thing that you can do if this is reproducible with Nightly is to capture a performance profile. Please note that you need to use Nightly version to be able to capture the performance profile. You can get more info on how to install and use the Cleopatra add-on (that helps you get the performance profile) by going to:
https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Profiling_with_the_Built-in_Profiler
https://perf-html.io/

Comment 2

[sworddragon2]

On trying to retest this issue I'm not able to reproduce it with the same version of Firefox and with the same profile. I tried several streams on Twitch and a recording of the video where I saw this issue (all with the video quality being set to source quality) but the infinite memory claiming did not happen anymore. Since I did update my system in the meantime either something else caused this issue (like a library) which is now possibly fixed or there is a specific condition which did not met anymore on my retest.

If I should see this issue again I continue the tests.

Comment 3

[ovidiu boca[:Ovidiu]]

Thanks, sworddragon2 for your reply.
I will close this bug as resolved incomplete, but please feel free to reopen it if you see this issue again.

Comment 4

[sworddragon2]

Over the past after my last post I saw this issue still very often. As I also switched to Windows 10 in this time this issue is not limited to Linux.

There are also some reports about memory leaks on Twitch here but none that I have found seem to be really the issue I'm seeing experiencing or the tickets are mainly focussing on CPU issues and just mentioning memory issues slightly. But here is the behavior I have observed so far (might be very slightly off from my memory or changes in Firefox over the course of time):

• Closing all tabs and leaving Firefox alone for a while (hours up to the next day) after accumulating several GiB of memory via this issue causes Firefox to keep the memory for usually several hours but it might be released after several hours later (like the next day).
• Going to about:memory and minimizing the memory usage does free the several GiB of memory.
• This troublesome memory accumulation of Firefox has a very good chance triggering Windows's Resource Exhaustion Detector (which happened here already several times) which then usually kills multiple processes causing potential data loss (even out of Firefox).
• There are many people blaming on this issue on the internet (like reddit) and some information hints that the memory accumulating here might actually come from the chat instead of the video.

The first 2 points imply that this issue is not a hard memory leak as Firefox still can reach the allocated memory at any time. It might be some sort of cache or something similar that is just not being released in a sane manner. Eventually this hint makes this issue much easier to fix.

The last point implies that this issue might also affect many other websites (but just on a very much lower level as they do usually not accumulate information on idle like a Twitch stream does). If this is the case it might be even more worth investigating this issue as it might significantly improve Firefox's overall memory usage.

Comment 5

[sworddragon2]

On Firefox 73.0.1 I did now test this on a new profile to figure more out:

• With a new profile without changing anything the issue doesn't reproduce.
• I figured out that the combined settings of enabling private browsing mode and setting privacy.resistFingerprinting to true causes this issue to reproduce (but only with a chance of 10%-20% on opening a stream). I did also test a while against each of both settings individually but could never reproduce the issue this way.
  • In private browsing mode one of the Firefox processes increased its memory usage continuously when watching a stream on Twitch. It started at around ~130 MiB and increased at a rate of ~1 MiB/sec until ~180+ MiB where it did reset to its initial value of ~130 MiB when the issue did not reproduce. As a note the actual rate of memory increasement was dependent on the video quality. The rate of 1 MiB/sec was achieved on 1080p (which is the source quality for most streams) which utilized on average 8 Mibit/sec network traffic.
  • The above implies the increasement of memory could be actually the caching of the stream in memory since the memory increasement matches the network traffic. Disabling permanent private browsing mode causes the related Firefox process to not increase its memory and thus stalling at ~150 MiB. However, permanent disk activities occur at this time possibly due to caching the stream on disk instead of memory and thus supporting the above theory that the increase in memory is from caching the stream into memory. Additionally once the issue appears going to about:memory and clicking GC and CC do not significantly decrease the memory usage (only around a few MiB) but only minimizing the memory usage will.
  • Setting privacy.resistFingerprinting to true has now a good chance to prevent the process of resetting the memory (assumingly flushing the cache). If this issue does not occur on a stream simply reloading the site can change this (the already mentioned 10%-20% chance).

Also here are the new STR based on the above information:

1. On a new profile enable permanent private browsing mode and set privacy.resistFingerprinting to true and restart the browser.
   1.1 If Firefox asks you at any time now if you want to request the english version of websites answer No (however, the choice here probably doesn't matter).
2. Go to https://www.twitch.tv and open any stream.
   2.1. If the stream notifies you that the content is being targeted at adults click that you still want to watch it.
3. In the Twitch video player set the video quality to the highest possible value (recommended 1080p).
4. Open the task manager and go to the Details register. Look for a Firefox process that increases constantly its commited memory. It starts on around 130 MiB and increases about 1 MiB/sec (if 1080p have been chosen previously).
5. If this Firefox process does increase its memory to only around ~180-200 MiB and then resets to ~130-150 MiB or if you see many drops of just a few MiB the bug did not trigger. In this case simply click the reload button to reload the site. Repeat this step until you don't see those memory drops anymore (might need 5-10 tries (or more if very unlucky)) and the Firefox process accumulates 500 MiB of memory as safety.
6. Close all tabs and wait some minutes and then check the memory usage of this Firefox process again.
7. Go to about:memory, click "Minimize memory usage" and check the memory usage of this Firefox process again.

After step 6 the memory is still not being freed but after step 7 it is.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.