RPCS3 2017 Wrap-up: A Stunning Year of Progress

Introduction

RPCS3 is an open source Sony Playstation 3 emulator for Windows and Linux. Currently, it has two full-time paid developers, and numerous volunteer developers. RPCS3 has existed since 2011, but development took off a year ago when the Patreon launched and Nekotekina followed by kd-11 began working on the emulator full time. This post will summarize what interesting developments and progress has happened in the last year. If you want to see how the codebase has changed over time, you can head over to the full list of merged PRs for 2017 and check it out. As seen on the GitHub statistics page we had 630 pull requests merged during 2017, and many of these contain numerous commits. That is a lot of code!

RPCS3? What Does That Even Mean?

When you think about it there is a tradition of Sony emulator names in this style. You have ePSXe, PCSX2, JPCSP, PPSSPP, and now RPCS3 as examples of PS1, PS2, PSP, and PS3 emulators. There is a story behind all of these names of course, but in the case of this little PS3 emulator it was given this name by the founder DH (who no longer works on the project). It is an abbreviation and combination of Personal Computer (PC) and PlayStation 3 (PS3). PC + PS3 = PCS3. What does the R stand for? DH and BlackDemon recollect that it used to stand for Real as back in 2011 there were only fake malware emulators around. Then at some point DH who is from Ukraine started saying the R stood for Russian which is what we roll with today as Nekotekina who joined in late 2013 and quickly became one of the biggest developers and the architect, is from Russia. So RPCS3 stands for Russian Personal Computer Station 3.

In this image of the old branding from 2016 RPCS3 is written with the Russian letter я, though this letter is in fact not pronounced like a Latin R at all (it is [ja] as in “yard”).

A short introduction

Where Were We One Year Ago?

One year ago in January of 2017 RPCS3 development had stagnated. The emulator itself ran almost no games properly, only roughly 20 titles were playable, and most of these were simple 2D games. Quite a few more did show intros or even go ingame, but due to generally very poor performance, bugs, and crashes almost nothing was really playable.

January 15th, 2017, tkoham opened an issue suggesting that RPCS3 launch a Patreon. A long discussion starting on GitHub and moving on over to a new and at the time “private” Discord channel and a few days later, the Patreon was launched. Shortly after, the Discord server went public and the new website launched. At the time, only a handful of commercial games even booted, and only the simplest of games, such as the Arkedo series, were anything near playable. The UI was unwieldy, installing games was tedious, and manually picking LLE modules was an immense challenge. Development had slowed considerably, with only Nekotekina and a handful of others making occasional commits.

Continue reading RPCS3 2017 Wrap-up: A Stunning Year of Progress

Progress Report: December 2017

What a way to end the year, eh? We started the last month of 2017 with a bang with RPCS3 taking major steps towards emulating many heavy hitting AAA titles, including quite a few exclusives like Uncharted and God of War. Some moved in to Loadable, others Ingame, and a few are even playable now. More can be read about this here. While these changes were made last month and received a post of their own, they had an impact on many other games that were discovered this month, and such incredible work deserves recognition. This progress report will focus on progress over the month of December. First and foremost, below is a video summary and report of various games that improved this month. Take a look, and read on for even more detail!

Table of Contents

Major Improvements
Games
Commits
Upcoming
Conclusion

First of all, let us have a look at the compatibility database statistics. As we predicted last month the number of games in the Nothing category is now less than 100! A year ago this with the Loadable and Intro categories represented the biggest chunk of registered games, most simply did not work more or less.
Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (October 2017)

However we are now making quite a drastic change to how compatibility is counted. Above different versions of the same game are counted separately. So for example instead of simply having one entry for Demon’s Souls, there are currently four entries: the european disc and PSN versions, and the american disc and PSN versions. This could even become six entries if someone were to test the japanese counterparts.

This is a very traditional way for emulators to look at compatibility, but because of the nature of such a “modern” console like PS3, there can be a lot of different versions of the same game. We have different regions: Europe, America, Japan, Asia, Korea, Hong Kong. And for each region the disc version and the PSN version.

So as can be seen below, we are merging all entries of the same game under the same media type (disc or digital), into a single entry.
There are some differences on compatibility between media types, like Demon’s Souls, where only the disc versions are currently playable, while the PSN versions fail to get past a black screen. But every disc version is playable the same, and every PSN version fails the same, so it makes sense to simply count the disc versions Playable once, and the PSN versions as Nothing once in the statistics.
Also note that not every game is merged yet, so the entries count is likely to go down again in the next month.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (October 2017)

Here is the full log of every game that changed status in December

Speaking of statistics, over on github 13 authors have pushed 95 commits where roughly 115 files were changed, with 5000 lines added, and 1400 lines removed.

Continue reading Progress Report: December 2017

Progress Report: November 2017

November was one of the most interesting months of the year thanks to the huge amount of games that jumped out of Nothing into the Ingame/Playable categories. This is in part thanks to the hard work announced in December 1st, which made many previously non-working big titles now start working on RPCS3. Of course, some of the progress towards this announcement was merged gradually during the month of November. In this Progress Report, major emulation improvements will be detailed, followed by a list of noteworthy, interesting, and representative games that improved from these, and lastly individual contributions will be looked at closer.

Table of Contents

Major Improvements
Games
Commits
Closing Words

First things first, the compatibility statistics. In November we saw the category of games that do absolutely Nothing go down by over 100 games, and this while not taking in account any of the games that had reported improvements in December!
We are therefore making a prediction right now: thanks to the SPU/RSX fixes, amongst several other improvements: the number of known games that do Nothing will be less than 100 before the end of the year!

That is quite the feat because when the RPCS3 project got revitalized in January, thanks to Patreon and the full time employment of Nekotekina we had less than 100 games that were Playable – that to quite a lower standard than today. And the category Nothing didn’t even exist because we argued that “What is the point? Almost every game does Nothing anyway”. How far have we come!

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (November 2017)

Looking at the GitHub statistics for November 16 authors have pushed 172 commits with roughly 11,000 added lines of code, and 5,000 deletions.

Continue reading Progress Report: November 2017

Introducing High Resolution Rendering

One of the most anticipated features has just been added to RPCS3! High resolution rendering allows users to play at resolutions far exceeding what the PS3 could handle. If you thought your favourite PS3 games were starting to look a bit dated, just wait until you get to experience them in up to 10k! Although, we doubt many users will have the setup necessary to benefit from 10k today, emulation is all about preserving for tomorrow.

The video above showcases various popular titles with a side-by-side comparison from 720p (native PS3 resolution) to 4k rendering with 16x anisotropic filtering in RPCS3. The difference is quite incredible in titles that have high quality assets. There is a lot of detail that just wasn’t visible at 720p. This feature is available for every PlayStation 3 game running on RPCS3. The only exception being that it does not yet work with Strict Rendering Mode. Games that require this setting will have to wait a bit longer before they can benefit from high resolution rendering.

Performance

Rendering a modern PC game in high resolutions such as 4k, while beautiful, is quite taxing on your hardware and there is often a massive hit in performance. However, since most of the workload for RPCS3 is on the CPU and GPU usage is low, there is a lot of untapped performance just waiting to be used. All processing is done CPU side, and as far as the GPU is concerned it is simply rendering 2006 era graphics (yes, the PS3 is 11 years old now). We’re happy to report that anyone with a dedicated graphics card that has Vulkan support can expect identical performance at 4k.

Anisotropic Filtering (AF)

High resolution support wasn’t the only thing that was added in this update! Another reason for such a massive upgrade in visual fidelity is having full 16x AF support. This greatly improves how textures can look, especially when viewed at an angle. Take a look at the below screenshots of Ni no Kuni as an example of the default AF vs forced 16x. The difference is especially noticeable on the ground inside the gate.

Ni No Kuni with default AF (left) and forced 16x AF (right)

Continue reading Introducing High Resolution Rendering

Progress Report: September 2017

September was quite the exciting month. Our lead graphics developer, kd-11, made major improvements to graphics emulation while Nekotekina, among many things, implemented initial networking support. This progress report will provide an overview of major changes and games that improved.

Table of Contents

Major Improvements
Games
Commits
Upcoming
Conclusion

First of all, here are the compatibility statistics for this month. As usual the category of games that do nothing is shrinking while more and more games become playable.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (September 2017)

Looking at the GitHub statistics, 15 authors have pushed 131 commits to the master branch. Here 216 files have changed and there have been 9,185 additions and 7,028 deletions of lines of code. Below some of the major improvements from these code changes are summarized.

Continue reading Progress Report: September 2017

Progress Report: August 2017

August was a month of bug fixes and rewriting old code. In this report some general emulation improvements will be explained, followed by a closer look at some popular and representative games. Lastly, individual contributions will be covered and plans for the future will be detailed.

Table of Contents

Improvements
Games
Commits
Upcoming
Conclusion

First and foremost are the compatibility numbers for August. Finally there are more games that are playable than completely not working! In fact, less than a quarter of all tested games do not at least show something on the screen. Just a year ago there wasn’t even a compatibility database or listing of games that did nothing as almost every games was expected to do nothing or at best be loadable! A lot of the improved games are actually the result of LLE GCM from July as testing and updating the forums and the compatibility database took quite some time. That said, a lot of games improved from new changes this month, which will be covered below.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (June 2017)

Looking at the GitHub statistics, 20 authors have pushed 179 commits to the master branch. Here 190 files have changed and there have been 8,587 additions and 3,876 deletions of lines of code. Below some of the major improvements from these code changes are summarized.

Continue reading Progress Report: August 2017

Rewriting Vertex Processing for Massive Performance Gains

Greetings. I am kd-11, graphics developer for rpcs3 with a mid-month update on latest developments on the emulator.

As many are already aware, a lot has been going on lately with the new changes to the RSX (the PS3 GPU) emulation, dubbed vertex rewrite. This change moves a lot of vertex processing duties from the CPU to the GPU where they rightly belong and as a result there are massive performance gains especially with OpenGL but also with Vulkan in geometry heavy scenes.

Background

Most if not all users are probably aware by now, but dedicated graphics cards exist on a physically separate board. This means data has to be moved to and from it through the PCI-E bus which is quite fast. However, while it is high bandwidth, it is also high-latency. That means you cannot just send something over there and expect to get it immediately available for the next draw call. Instead, the GPU has to wait for data to be prepared and then signaled that data is ready for processing before drawing begins. This is a general simplification, but it helps illustrate the point. The RSX on the PS3 doesn’t work the same way however. It has near direct access to the XDR main memory on a PS3 and ‘pulls’ data directly from main memory as though it were local memory. It is somewhat similar to integrated graphics memory in this case. That means data is not ‘pre-packaged’ for transport to the PS3 GPU since the memory is virtually unified from the point of view of the RSX. When using Vulkan, drawing is not scheduled until the whole command queue is flushed mitigating the impact of transfer since data will likely have been uploaded beforehand, but for OpenGL this was a big bottleneck.

The second issue was that the emulator was doing a lot of computation on the CPU on how to read vertex data from main memory, essentially pre-packaging the data into formats easy for GPUs to use. This is a very slow process and also very memory intensive (hence the ‘Working buffer not enough’ crashes). Enabling a debug overlay with the old method shows some games taking up to 200ms to prepare vertex data for one frame (Hellboy: The Science of Evil). This is obviously not optimal. The impact could be lowered by using more threads for vertex processing, but with the number of threads already needed to emulate the PS3’s multi-core processor, it was a problem. Spawning 8+ vertex processing threads reduced the time spent processing vertices, but cost other threads to starve and performance would drop significantly. The solution was to shift the work to the GPU instead and not touch it in any way. Just copy the data block and the GPU could fetch the data it needed for itself, mimicking the behaviour of the real hardware.

Continue reading Rewriting Vertex Processing for Massive Performance Gains

Progress Report: July 2017

July like every month before it this year set a new record in the number of improvements that happened. Mostly centered around bug fixes and compatibility improvements it is safe to say that if every single improvement were to be covered in great detail this progress report would take more than a month to finish. Therefore the format is now going to change a bit. This report will focus on some major emulation improvements and it will explain what these entail in general. Thereafter a few select more interesting games and how they were improved will be covered. Every improved game will not be covered because there were simply too many, and evaluation of earlier reports indicate that it isn’t interesting content either.

First of all are the compatibility database statistics for the month of July. Take note that the last database update was performed a day before the major emulation improvement known as “LLE gcm” was merged, meaning the hundreds of games improved from this are not listed in the figures below, or even on the compatibility database yet.

Table of Contents

Improvements
Games
Commits
Upcoming
Conclusion

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (June 2017)

Looking at the GitHub statistics, 18 authors have pushed 201 commits to the master branch. Here 257 files have changed and there have been 14,559 additions and 5,088 deletions of lines of code. What improvements came from these changes? Let’s take a look:

Continue reading Progress Report: July 2017

RPCS3 AppImages are now available for Linux!

Nearly six months after the Patreon launched the RPCS3 team have finally improved RPCS3 on Linux to the point where it has reached compatibility and stability parity with Windows. Thanks to the hard work by hcorion we can finally start to provide pre-compiled binaries in the form of AppImages for easy installation on your favorite distribution.

What Was the Issue?

There were a lot of problems. Back in January, quite literally nothing was working. RPCS3 would crash instantly upon booting any game, if the program itself would even start at all. Moreover, additional functionality like the debugger, framerate counter, and firmware installer were completely broken too. While many of these auxiliary issues were quickly identified and fixed, the fact of the matter was that almost every game would hang after running for a few seconds. This turned out to be much more difficult to fix. This was caused by several different bugs in thread synchronization which were fixed continuously in the past few months. Finally, one last relatively small commit in early April fixed the last bug and suddenly RPCS3 on Linux went from basically nothing to Demon’s Souls and [redacted]. Or so we though, but we quickly found out that the LLVM recompiler was completely broken for a lot of users who just got completely nonsense errors. We encountered strange and esoteric bugs and oddities about LLVM and how various Linux software, including the Mesa drivers, were using it. These problems made RPCS3 unusable for a lot of people. A lot of false flags and red herrings later the bugs were fixed not by changing any code, but by using rare compiler flags. RPCS3 on Linux is now working as intended for everyone, including AMD and Intel graphics users with modern Mesa. Even Vulkan with Mesa is now working!

PlayStation 3 Games on Linux

Below are some popular PlayStation 3 games showcased running on Linux. Performance is about the same as on Windows, perhaps even a few percent better in some very intensive games like [redacted]. But take note: These images were either captured on a laptop with a very old i7-2670qm CPU, or a fast desktop with a i7-4770 CPU.

Continue reading RPCS3 AppImages are now available for Linux!

Progress Report: June 2017

Another month another colossal RPCS3 progress record. But first things first, take a look at this beautiful introduction to RPCS3 video that reznoire made:

[redacted]

Moving on to some statistics the numbers alone this month are quite impressive, and that is not even counting games that improved from emulation improvements that are still work in progress.
Game Compatibility: Game Status

Game Compatibility: Monthly Improvements (June 2017)

Looking at the GitHub statistics 16 authors have pushed 140 commits to the master branch. Here 353 files have changed and there have been 19,729 additions and 17,660 deletions. These numbers are much larger than normal, and several exciting changes are behind it. This month RPCS3 transitioned away from the previous GUI toolkit Wx to Qt which in turn also lead to several user facing interface improvements. This separate report goes into significantly greater detail regarding what work has already been done on the GUI, and what work is planned to be done in the future. On a lower level, Nekotekina reworked the entire PPU LLVM recompiler to greatly increase compatibility of it. kd-11 fixed various graphical issues affecting hundreds of games, most notably broken shadows and depth of field in various titles. Moreover kd-11 also enabled Vulkan on Linux and pushed support for it to the master branch. Nekotekina and Numan also worked on .sprx relocations in general and in the LLVM recompiler which in simple terms is a compatibility and performance improvement.

The talented developer Jarves is working on a very important core improvement. It is not yet finished or merged to the master branch, but as work on this has been going on in public, and a lot of people have already tested it and submitted results this report will cover the work anyway. Jarves is working on “LLE gcm” which is huge and here is why:

LLE stands for Low Level Emulation. In RPCS3 this means that a PS3 operating system module is run directly as it is via lower level emulation methods. This is great for compatibility because it doesn’t involve much guesswork and is very accurate. As far as the games are concerned, they are working with the exact same operating system methods with the exact same implementations here as on a real PS3.

gcm or “graphics command management” is the part of the PS3 operating system responsible for creating and managing various graphics commands, including everything from how to set up vsync to graphics memory allocation.

LLE gcm is huge. A very core part of emulating games is now done exactly the same as on on a real PS3, and the result is amazing. For example Red Dead Redemption no longer hangs almost instantly. The Last of Us doesn’t crash immediately and actually shows a loading screen and goes through a lot of initialisation successfully. [redacted] no longer randomly crashes. Various high end games that did nothing before are now starting to boot, for example the Yakuza series where Yakuza 3 went from nothing to ingame, and Yakuza 4 and 5 went from nothing to loading screens. One could go on and on about what a substantial compatibility improvement LLE gcm is (and I will). Now combine this with the greatly improved LLVM recompiler and over 40 commits of graphics fixes and you have a month of quite insane progress. This would be a great segway into looking at improved games and such, but Jarves did one other important contribution that deserves its own section.

Continue reading Progress Report: June 2017