Rewriting Vertex Processing for Massive Performance Gains

Greetings. I am kd-11, graphics developer for rpcs3 with a mid-month update on latest developments on the emulator.

As many are already aware, a lot has been going on lately with the new changes to the RSX (the PS3 GPU) emulation, dubbed vertex rewrite. This change moves a lot of vertex processing duties from the CPU to the GPU where they rightly belong and as a result there are massive performance gains especially with OpenGL but also with Vulkan in geometry heavy scenes.

Background

Most if not all users are probably aware by now, but dedicated graphics cards exist on a physically separate board. This means data has to be moved to and from it through the PCI-E bus which is quite fast. However, while it is high bandwidth, it is also high-latency. That means you cannot just send something over there and expect to get it immediately available for the next draw call. Instead, the GPU has to wait for data to be prepared and then signaled that data is ready for processing before drawing begins. This is a general simplification, but it helps illustrate the point. The RSX on the PS3 doesn’t work the same way however. It has near direct access to the XDR main memory on a PS3 and ‘pulls’ data directly from main memory as though it were local memory. It is somewhat similar to integrated graphics memory in this case. That means data is not ‘pre-packaged’ for transport to the PS3 GPU since the memory is virtually unified from the point of view of the RSX. When using Vulkan, drawing is not scheduled until the whole command queue is flushed mitigating the impact of transfer since data will likely have been uploaded beforehand, but for OpenGL this was a big bottleneck.

The second issue was that the emulator was doing a lot of computation on the CPU on how to read vertex data from main memory, essentially pre-packaging the data into formats easy for GPUs to use. This is a very slow process and also very memory intensive (hence the ‘Working buffer not enough’ crashes). Enabling a debug overlay with the old method shows some games taking up to 200ms to prepare vertex data for one frame (Hellboy: The Science of Evil). This is obviously not optimal. The impact could be lowered by using more threads for vertex processing, but with the number of threads already needed to emulate the PS3’s multi-core processor, it was a problem. Spawning 8+ vertex processing threads reduced the time spent processing vertices, but cost other threads to starve and performance would drop significantly. The solution was to shift the work to the GPU instead and not touch it in any way. Just copy the data block and the GPU could fetch the data it needed for itself, mimicking the behaviour of the real hardware.

Continue reading Rewriting Vertex Processing for Massive Performance Gains

Progress Report: July 2017

July like every month before it this year set a new record in the number of improvements that happened. Mostly centered around bug fixes and compatibility improvements it is safe to say that if every single improvement were to be covered in great detail this progress report would take more than a month to finish. Therefore the format is now going to change a bit. This report will focus on some major emulation improvements and it will explain what these entail in general. Thereafter a few select more interesting games and how they were improved will be covered. Every improved game will not be covered because there were simply too many, and evaluation of earlier reports indicate that it isn’t interesting content either.

First of all are the compatibility database statistics for the month of July. Take note that the last database update was performed a day before the major emulation improvement known as “LLE gcm” was merged, meaning the hundreds of games improved from this are not listed in the figures below, or even on the compatibility database yet.

Table of Contents

Improvements
Games
Commits
Upcoming
Conclusion

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (June 2017)

Looking at the GitHub statistics, 18 authors have pushed 201 commits to the master branch. Here 257 files have changed and there have been 14,559 additions and 5,088 deletions of lines of code. What improvements came from these changes? Let’s take a look:

Continue reading Progress Report: July 2017

RPCS3 AppImages are now available for Linux!

Nearly six months after the Patreon launched the RPCS3 team have finally improved RPCS3 on Linux to the point where it has reached compatibility and stability parity with Windows. Thanks to the hard work by hcorion we can finally start to provide pre-compiled binaries in the form of AppImages for easy installation on your favorite distribution.

What Was the Issue?

There were a lot of problems. Back in January, quite literally nothing was working. RPCS3 would crash instantly upon booting any game, if the program itself would even start at all. Moreover, additional functionality like the debugger, framerate counter, and firmware installer were completely broken too. While many of these auxiliary issues were quickly identified and fixed, the fact of the matter was that almost every game would hang after running for a few seconds. This turned out to be much more difficult to fix. This was caused by several different bugs in thread synchronization which were fixed continuously in the past few months. Finally, one last relatively small commit in early April fixed the last bug and suddenly RPCS3 on Linux went from basically nothing to Demon’s Souls and [redacted]. Or so we though, but we quickly found out that the LLVM recompiler was completely broken for a lot of users who just got completely nonsense errors. We encountered strange and esoteric bugs and oddities about LLVM and how various Linux software, including the Mesa drivers, were using it. These problems made RPCS3 unusable for a lot of people. A lot of false flags and red herrings later the bugs were fixed not by changing any code, but by using rare compiler flags. RPCS3 on Linux is now working as intended for everyone, including AMD and Intel graphics users with modern Mesa. Even Vulkan with Mesa is now working!

PlayStation 3 Games on Linux

Below are some popular PlayStation 3 games showcased running on Linux. Performance is about the same as on Windows, perhaps even a few percent better in some very intensive games like [redacted]. But take note: These images were either captured on a laptop with a very old i7-2670qm CPU, or a fast desktop with a i7-4770 CPU.

Continue reading RPCS3 AppImages are now available for Linux!

Progress Report: June 2017

Another month another colossal RPCS3 progress record. But first things first, take a look at this beautiful introduction to RPCS3 video that reznoire made:

[redacted]

Moving on to some statistics the numbers alone this month are quite impressive, and that is not even counting games that improved from emulation improvements that are still work in progress.
Game Compatibility: Game Status

Game Compatibility: Monthly Improvements (June 2017)

Looking at the GitHub statistics 16 authors have pushed 140 commits to the master branch. Here 353 files have changed and there have been 19,729 additions and 17,660 deletions. These numbers are much larger than normal, and several exciting changes are behind it. This month RPCS3 transitioned away from the previous GUI toolkit Wx to Qt which in turn also lead to several user facing interface improvements. This separate report goes into significantly greater detail regarding what work has already been done on the GUI, and what work is planned to be done in the future. On a lower level, Nekotekina reworked the entire PPU LLVM recompiler to greatly increase compatibility of it. kd-11 fixed various graphical issues affecting hundreds of games, most notably broken shadows and depth of field in various titles. Moreover kd-11 also enabled Vulkan on Linux and pushed support for it to the master branch. Nekotekina and Numan also worked on .sprx relocations in general and in the LLVM recompiler which in simple terms is a compatibility and performance improvement.

The talented developer Jarves is working on a very important core improvement. It is not yet finished or merged to the master branch, but as work on this has been going on in public, and a lot of people have already tested it and submitted results this report will cover the work anyway. Jarves is working on “LLE gcm” which is huge and here is why:

LLE stands for Low Level Emulation. In RPCS3 this means that a PS3 operating system module is run directly as it is via lower level emulation methods. This is great for compatibility because it doesn’t involve much guesswork and is very accurate. As far as the games are concerned, they are working with the exact same operating system methods with the exact same implementations here as on a real PS3.

gcm or “graphics command management” is the part of the PS3 operating system responsible for creating and managing various graphics commands, including everything from how to set up vsync to graphics memory allocation.

LLE gcm is huge. A very core part of emulating games is now done exactly the same as on on a real PS3, and the result is amazing. For example Red Dead Redemption no longer hangs almost instantly. The Last of Us doesn’t crash immediately and actually shows a loading screen and goes through a lot of initialisation successfully. [redacted] no longer randomly crashes. Various high end games that did nothing before are now starting to boot, for example the Yakuza series where Yakuza 3 went from nothing to ingame, and Yakuza 4 and 5 went from nothing to loading screens. One could go on and on about what a substantial compatibility improvement LLE gcm is (and I will). Now combine this with the greatly improved LLVM recompiler and over 40 commits of graphics fixes and you have a month of quite insane progress. This would be a great segway into looking at improved games and such, but Jarves did one other important contribution that deserves its own section.

Continue reading Progress Report: June 2017

RPCS3 Has Moved to Qt

June has been an exciting month for RPCS3. Quite a few new features have been added to the emulator, with a healthy focus on the somewhat outdated GUI. This post will break down some of changes brought by Qt, as well as new features introduced since the transition. This mini progress report will only focus on GUI changes. The main progress report to be published tomorrow will cover the rest.

First up is, obviously, the actual transition from the user interface toolkit WxWidgets to Qt. This transition has been a long and time consuming process, but it added a host of new functionality, both visible and not so visible. A non-exhaustive list of the transition pull requests, the main of which can be found here is:

  • – Appveyor and Travis now build with Qt project, thanks to hcorion. This switches the nightly builds, and eventually the linux builds, to use the Qt interface.
  • – Made some design changes to the GUI, such as progress bars and taskbars now showing percent completion and slight improvements to the Vulkan/DX12 adapter selection box to make it easier to use.
  • – Added support for layouts, which allows you to move your docked widgets as you please.
  • – Added support for booting games in fullscreen.
  • – Added a Welcome screen to point new users to the Quickstart setup guide.
  • – Support for themes! An example can be found further down.

As with all major changes, this caused a few hiccups, but they are being worked on as they are found, and issues can be reported here.

This transition to Qt brought a wave of GUI improvements with it. First up was a recent games list, implemented in #2843 and #2847. This saves the last nine games launched in an easy to use list that has hotkeys, so that those with large libraries, or those using Blu-ray disc games, can easily launch their favorites. The list can be frozen, though items cannot yet be pinned.

Second was a new viewing mode and a tool bar. The game list can now be viewed as a simple grid:

As you can see in the screenshot provided by user Talkashie(which also shows a custom made theme!), the new toolbar allows for searching of large libraries, as well as directly filtering the list by game types. An example being that you can filter out audio/video apps and home apps, or you can choose to view only HDD games. Thanks to user rutantan for creating the very nice button icons in the toolbar.

Continue reading RPCS3 Has Moved to Qt

Progress Report: May 2017

May was a very eventful month for RPCS3 as we saw significant core and performance improvements. The goal of this progress report is to highlight some of the more notable or interesting developments of the project. The report will start by showcasing a selection of games that were improved in one way or another. Thereafter we will summarize what work each contributor did this month.

Game Compatibility: Game Status

Game Compatibility: Monthly Improvements (April 2017)

kd-11 joined Nekotekina’s Patreon

You can read about this in more detail here.
The short version is that kd-11 is an extremely talented graphics developer who has helped RPCS3 since January of 2016. Without his work RPCS3 would not be here today. It goes the other way around too of course, without the significant core accuracy and performance improvements by Nekotekina we would not be here today either. Therefore it make sense for two great minds to join forces. With the generous Patreon support kd-11 will be able to acquire new hardware for development and testing. On the short term purchasing list is a modern NVIDIA GPU, a modern AMD GPU, and possibly a Skylake+ laptop of some kind. This would allow kd-11 to fix specific issues on these platforms. Not many people use RPCS3 with integrated Intel graphics but they are technically fast enough (for now at least). The problems lie in very very buggy Intel graphics drivers which is why it would help to have direct access to it.

Check out this video below, it highlights some games that saw performance improvements from “the secret build” which was a huge general graphics rework, with focus in particular on Vulkan.

Continue reading Progress Report: May 2017

Lead Graphics Developer kd-11 Is Joining Nekotekina’s Patreon

Veteran RPCS3 graphics developer kd-11 is joining Nekotekina’s Patreon. kd-11 has worked on RPCS3 since January 2016 and has since then fixed countless bugs in all graphics renderers and implemented a lot of missing functionality. Without his dedicated work, RPCS3 would certainly not be where it is today.

As you know lead core developer Nekotekina, who has worked on RPCS3 since 2013, launched his highly successful Patreon in January this year. Thanks to the overwhelming amount of support, he has been able to work on RPCS3 full time for 5 months now and the progress he has done is staggering. The RPCS3 team has discussed the best course of action for the road forward and have decided that the time is now right for kd-11 to join Nekotekina’s Patreon in order for him to obtain hardware necessary for development and testing.

The Purpose

As kd-11 works on the graphics side of things, especially the renderers using OpenGL, Vulkan, and D3D12 (Direct3D 12), modern hardware is required for development and testing. Ideally, he needs a modern AMD, Nvidia, and Intel GPU to easier identify bugs specific to one platform. Furthermore, kd-11 works as a consultant and therefore has the possibility to accept or turn down certain jobs. If support on Patreon reaches a certain level, he may be able to spend more time on the project.

As of now the goal is obtaining necessary hardware for development and testing:

  • – A modern AMD GPU that supports DirectX12 and Vulkan.
  • – A modern Nvidia GPU that supports DirectX12 and Vulkan.
  • – A modern Intel CPU where the iGPU supports DirectX12 and Vulkan

Roadmap

With the support of generous patrons kd-11 will be able to work on the following tasks:

Short term

  • – Investigate game specific bugs like the overly bright bloom in [redacted] or the overly dark lighting in Demon’s Souls.
  • – Improve performance and compatibility of all renderers.
  • – Enable Vulkan on Linux, which will bring a tremendous performance improvement for some games.
  • – Implement missing functionality that will fix various bugs in different games. For example, the broken shadows in many games including Demon’s Souls depend on this task.

Mid term

  • – Properly support some PlayStation 3 features like MSAA that are lacking at the moment.
  • – Implement RSX reports which are missing at the moment.

Long term

  • – Enable higher rendering resolution to play games at any resolution, for example 4K instead of 720p in Demon’s Souls or [redacted].
  • – Enable extra graphics options to support extra rendering features, such as custom anti-aliasing modes.

Continue reading Lead Graphics Developer kd-11 Is Joining Nekotekina’s Patreon

Progress Report: April 2017

April was a very eventful month for RPCS3. The goal of this progress report is to highlight some of the more notable or interesting developments of the project. The report will start by showcasing a selection of games that were improved in one way or another. Thereafter we will summarize what work each contributor did this month.

Since the last progress report, approximately 18 authors have made 104 commits, added 9,621 new lines of code and deleted 1,904 lines of code.

Game Compatibility: Game Status

Game Compatibility: Monthly Improvements (April 2017)

[redacted]

This is the last real PlayStation 3 game and it also happens to be highly popular. Let us take a look at a few things.

People often ask what kind of CPU is needed to run this game. The answer is that no CPU is truly fast enough right now, but if you enjoy playing games at 10 FPS or so, then feel free to get a CPU with a lot physical cores. See the screenshot below for the motivation behind this statement. Of course in the future when RPCS3 performance is improved such an extreme CPU likely will not be required.

If you didn’t see it already, we made a video of [redacted] running in RPCS3 for our YouTube channel.

[redacted]

Here are some really interesting results. [redacted] proved that this game is locked to 30 FPS on PlayStation 4 as well as 30 FPS on PlayStation 3. However, in RPCS3 it is possible to unlock the framerate meaning that It is theoretically possible to run this game at more than 30 FPS. Look at the screenshot below, the game is really running at 40 FPS here with no problems. After all, if it can run fine at 15 FPS, half its intended frame rate, why would it not run fine at 60 FPS, double its intended frame rate?

Maybe there are some bugs, but maybe these could be patched just like Nekotekina patched an engine bug in Disgaea. Digital Foundry also pointed out the only other difference between the PlayStation 3 and the PlayStation 4 version of the game is the rendering resolution. But it is theoretically possible to increase internal rendering resolution in this game (although nothing has been implemented in RPCS3 yet). The point is, in the future the RPCS3 version of [redacted] could be the best version. [redacted] running at 4K and 60 FPS is very possible.

Continue reading Progress Report: April 2017

Progress Report: March 2017

March 2017 beat the previous record set by February 2017 as the most eventful month in the history of the project. So much happened that even this colossal progress report can only begin to scratch the surface.

First of all, lead RPCS3 developer Nekotekina reached the $1000 goal on Patreon, securing his long time commitment to work on RPCS3 full time. This was the direct result of the massive amount of attention the project got when two popular games were drastically improved this month. First, Demon’s Souls went from crashing in zero seconds to going ingame and almost being playable. Second, the cult classic Catherine received significant performance improvements and now runs with practically perfect graphics and performance. The two videos of these games received over 200 000 views on the official RPCS3 YouTube channel. Even the famous YouTube personality, TotalBiscuit was impressed with the progress, and the killing of the notorious Demon’s Souls boss Vanguard in RPCS3.

Of course the hundreds of videos posted by the community on YouTube, different forum posts, Reddit submissions, and so on, have contributed greatly to the massive growth of the RPCS3 community. In fact, new developer Inviuz aka Numan was one of these people who recently discovered RPCS3, spent a lot of hours reading the code and debugging Demon’s Souls, and finally getting it to boot for the first time. Oh and Red Dead Redemption also went to the main menu thanks to this, and some graphics improvements by kd-11. This showcases one of the most important strengths of RPCS3 being free and open-source software and it is very likely that more and more people will join the project in the future and contribute changes both big and small.

Since the last progress report, approximately 17 authors have made 115 commits with 6,831 lines of code added, and 4,471 lines of code deleted. And some significant improvements are still in the pipeline but have yet to be merged.

The progress report is mainly split into three different parts. First we will take a closer look at what each pull request this month did, and show a few practical examples. Thereafter we will take a look at a selection of some interesting games that were improved this month, though this is just a small slice of the hundreds upon hundreds of games that received small or significant improvements this month. Lastly we will take a look at some of the upcoming changes before rounding of this monthly progress report.

This Month’s Contributions

Let us start with Nekotekina who did some significant work this month. Over 60 commits encompassing a lot of major improvements such as automatic LLE module selection (probably the single most requested feature ever), LLVM 4 for Ryzen support, and staggering performance and stability improvements in many games.

Nekotekina’s Contributions

Automatic LLE (#2459)

Probably the single most anticipated feature ever. If you don’t know what this is, RPCS3 previously required you to manually pick which firmware modules should be loaded for each game. People would often have to refer to long lists of 20+ modules on the forums, often discovered by trial and error. However this is all ancient history now. RPCS3 will analyze the game executable and automatically load the required libraries now.

Reservations rewritten (#2404)

Substantial performance and stability improvements in many games. For example Catherine, [redacted], and Demon’s Souls saw large performance improvements from this. In fact, [redacted] is literally 10 times faster in all scenarios now, going from about 1 to about 10 FPS in the opening scene for example. In battles and dungeons [redacted] can peak at 30 FPS already. Catherine also runs at a stable 30 FPS in all scenarios with an overclocked Haswell-E 6-core or a Ryzen 7 8-core.

Continue reading Progress Report: March 2017

Progress Report: Demon’s Souls Nearly Playable and RPCS3 v0.0.2

It finally happened! The PlayStation 3 emulator RPCS3 has launched one of the most iconic games on the platform: Demon’s Souls. Are you curious to know how it all happened? What the developers went through? Welcome to this first of hopefully many to come developer logs where we will now tell you everything.

It all began in our Discord channel where a user going by the nickname, Numan approached us with some interesting news. He claimed to have conducted a detailed investigation of what Demon’s Souls was trying to do when it was crashing one second after booting. He drew the conclusion that the game was trying to allocate memory pages of different sizes, access levels, and different attributes. When the game then tried to access a specific memory address, it crashed with an access violation. Consequently, these discoveries lead to a more accurate implementation of the function sys_memory_get_page_attribute() which was responsible for this particular crash. Suddenly the game booted up in RPCS3 for the first time ever and showed a loading animation!

Continue reading Progress Report: Demon’s Souls Nearly Playable and RPCS3 v0.0.2