Progress Report: November 2017

November was one of the most interesting months of the year thanks to the huge amount of games that jumped out of Nothing into the Ingame/Playable categories. This is in part thanks to the hard work announced in December 1st, which made many previously non-working big titles now start working on RPCS3. Of course, some of the progress towards this announcement was merged gradually during the month of November. In this Progress Report, major emulation improvements will be detailed, followed by a list of noteworthy, interesting, and representative games that improved from these, and lastly individual contributions will be looked at closer.

Table of Contents

Major Improvements
Games
Commits
Closing Words

First things first, the compatibility statistics. In November we saw the category of games that do absolutely Nothing go down by over 100 games, and this while not taking in account any of the games that had reported improvements in December!
We are therefore making a prediction right now: thanks to the SPU/RSX fixes, amongst several other improvements: the number of known games that do Nothing will be less than 100 before the end of the year!

That is quite the feat because when the RPCS3 project got revitalized in January, thanks to Patreon and the full time employment of Nekotekina we had less than 100 games that were Playable – that to quite a lower standard than today. And the category Nothing didn’t even exist because we argued that “What is the point? Almost every game does Nothing anyway”. How far have we come!

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (November 2017)

Looking at the GitHub statistics for November 16 authors have pushed 172 commits with roughly 11,000 added lines of code, and 5,000 deletions.

Continue reading Progress Report: November 2017

RPCS3 Begins to Emulate Several Awaited AAA Exclusives. Here’s How We’ve Done It!

Yes, you’ve read correctly. Many of the much awaited exclusives are now finally starting to be emulated by RPCS3. In this blog post, you’ll learn which games we know to have improved and how we’ve done it.
But first, check out this awesome teaser:

Table of Contents

SPU Improvements by Jarves
RSX Improvements by kd-11
List of known Improved Games
Closing Words

SPU Improvements by Jarves

There were two main changes to the SPU emulation that brought us to this point of allowing so many newer titles to progress past ‘Intro.’ Let’s take a quick look at both individually.

SPU Interrupt Fix

Many titles in RPCS3 ‘hang’, but in the case of the titles mentioned above, they do not actually crash, and the fps counter would still change with just a black screen. This normally would be mistaken for RPCS3 just being slow and the game taking a bit to load, but opening up the debugger in RPCS3 tells a different story. The games would loop over the same code on both the PPU side and SPU side. In this case, they are waiting on something, but what?

‘The Last of Us’. More like – ‘The Last Loop That Will Ever Execute’
Moving the PC to address 0 of an SPU thread explains more. Most games will have just 0 (null) written there, but some actually have a branch there.

Continue reading RPCS3 Begins to Emulate Several Awaited AAA Exclusives. Here’s How We’ve Done It!

Progress Report: October 2017

October was a huge month for RPCS3. Our lead graphics developer, kd-11, implemented High Resolution Rendering and made many improvements to the emulator. While Nekotekina made some general improvements to the emulator, which will be discussed in this report, he also bought some parts and assembled a new computer with a future-proof CPU which will allow him to debug RPCS3 more efficiently, faster, and allow him to make sure newer instruction sets like AVX-512 work properly with RPCS3 when the right time comes. You can find more details on the latter and also discuss further on it at https://www.patreon.com/posts/pursuing-avx-512-15151688

Table of Contents

Major Improvements
Games
More Improvements (Commits)
Upcoming
Conclusion

Starting off with the compatibility statistics as usual, the Intro/Loadable/Nothing categories keep growing small and Ingame/Playable titles keep increasing. We’ve also hit 500 playable games this month!

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (October 2017)

Looking at the GitHub statistics, 16 authors have pushed 131 commits to the master branch. Here 180 files have changed and there have been 12,626 additions and 5,068 deletions of lines of code. Below some of the major improvements from these code changes are summarized.

Continue reading Progress Report: October 2017

Introducing High Resolution Rendering

One of the most anticipated features has just been added to RPCS3! High resolution rendering allows users to play at resolutions far exceeding what the PS3 could handle. If you thought your favourite PS3 games were starting to look a bit dated, just wait until you get to experience them in up to 10k! Although, we doubt many users will have the setup necessary to benefit from 10k today, emulation is all about preserving for tomorrow.

The video above showcases various popular titles with a side-by-side comparison from 720p (native PS3 resolution) to 4k rendering with 16x anisotropic filtering in RPCS3. The difference is quite incredible in titles that have high quality assets. There is a lot of detail that just wasn’t visible at 720p. This feature is available for every PlayStation 3 game running on RPCS3. The only exception being that it does not yet work with Strict Rendering Mode. Games that require this setting will have to wait a bit longer before they can benefit from high resolution rendering.

Performance

Rendering a modern PC game in high resolutions such as 4k, while beautiful, is quite taxing on your hardware and there is often a massive hit in performance. However, since most of the workload for RPCS3 is on the CPU and GPU usage is low, there is a lot of untapped performance just waiting to be used. All processing is done CPU side, and as far as the GPU is concerned it is simply rendering 2006 era graphics (yes, the PS3 is 11 years old now). We’re happy to report that anyone with a dedicated graphics card that has Vulkan support can expect identical performance at 4k.

Anisotropic Filtering (AF)

High resolution support wasn’t the only thing that was added in this update! Another reason for such a massive upgrade in visual fidelity is having full 16x AF support. This greatly improves how textures can look, especially when viewed at an angle. Take a look at the below screenshots of Ni no Kuni as an example of the default AF vs forced 16x. The difference is especially noticeable on the ground inside the gate.

Ni No Kuni with default AF (left) and forced 16x AF (right)

Continue reading Introducing High Resolution Rendering

Progress Report: September 2017

September was quite the exciting month. Our lead graphics developer, kd-11, made major improvements to graphics emulation while Nekotekina, among many things, implemented initial networking support. This progress report will provide an overview of major changes and games that improved.

Table of Contents

Major Improvements
Games
Commits
Upcoming
Conclusion

First of all, here are the compatibility statistics for this month. As usual the category of games that do nothing is shrinking while more and more games become playable.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (September 2017)

Looking at the GitHub statistics, 15 authors have pushed 131 commits to the master branch. Here 216 files have changed and there have been 9,185 additions and 7,028 deletions of lines of code. Below some of the major improvements from these code changes are summarized.

Continue reading Progress Report: September 2017

Progress Report: August 2017

August was a month of bug fixes and rewriting old code. In this report some general emulation improvements will be explained, followed by a closer look at some popular and representative games. Lastly, individual contributions will be covered and plans for the future will be detailed.

Table of Contents

Improvements
Games
Commits
Upcoming
Conclusion

First and foremost are the compatibility numbers for August. Finally there are more games that are playable than completely not working! In fact, less than a quarter of all tested games do not at least show something on the screen. Just a year ago there wasn’t even a compatibility database or listing of games that did nothing as almost every games was expected to do nothing or at best be loadable! A lot of the improved games are actually the result of LLE GCM from July as testing and updating the forums and the compatibility database took quite some time. That said, a lot of games improved from new changes this month, which will be covered below.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (June 2017)

Looking at the GitHub statistics, 20 authors have pushed 179 commits to the master branch. Here 190 files have changed and there have been 8,587 additions and 3,876 deletions of lines of code. Below some of the major improvements from these code changes are summarized.

Continue reading Progress Report: August 2017

Rewriting Vertex Processing for Massive Performance Gains

Greetings. I am kd-11, graphics developer for rpcs3 with a mid-month update on latest developments on the emulator.

As many are already aware, a lot has been going on lately with the new changes to the RSX (the PS3 GPU) emulation, dubbed vertex rewrite. This change moves a lot of vertex processing duties from the CPU to the GPU where they rightly belong and as a result there are massive performance gains especially with OpenGL but also with Vulkan in geometry heavy scenes.

Background

Most if not all users are probably aware by now, but dedicated graphics cards exist on a physically separate board. This means data has to be moved to and from it through the PCI-E bus which is quite fast. However, while it is high bandwidth, it is also high-latency. That means you cannot just send something over there and expect to get it immediately available for the next draw call. Instead, the GPU has to wait for data to be prepared and then signaled that data is ready for processing before drawing begins. This is a general simplification, but it helps illustrate the point. The RSX on the PS3 doesn’t work the same way however. It has near direct access to the XDR main memory on a PS3 and ‘pulls’ data directly from main memory as though it were local memory. It is somewhat similar to integrated graphics memory in this case. That means data is not ‘pre-packaged’ for transport to the PS3 GPU since the memory is virtually unified from the point of view of the RSX. When using Vulkan, drawing is not scheduled until the whole command queue is flushed mitigating the impact of transfer since data will likely have been uploaded beforehand, but for OpenGL this was a big bottleneck.

The second issue was that the emulator was doing a lot of computation on the CPU on how to read vertex data from main memory, essentially pre-packaging the data into formats easy for GPUs to use. This is a very slow process and also very memory intensive (hence the ‘Working buffer not enough’ crashes). Enabling a debug overlay with the old method shows some games taking up to 200ms to prepare vertex data for one frame (Hellboy: The Science of Evil). This is obviously not optimal. The impact could be lowered by using more threads for vertex processing, but with the number of threads already needed to emulate the PS3’s multi-core processor, it was a problem. Spawning 8+ vertex processing threads reduced the time spent processing vertices, but cost other threads to starve and performance would drop significantly. The solution was to shift the work to the GPU instead and not touch it in any way. Just copy the data block and the GPU could fetch the data it needed for itself, mimicking the behaviour of the real hardware.

Continue reading Rewriting Vertex Processing for Massive Performance Gains

Progress Report: July 2017

July like every month before it this year set a new record in the number of improvements that happened. Mostly centered around bug fixes and compatibility improvements it is safe to say that if every single improvement were to be covered in great detail this progress report would take more than a month to finish. Therefore the format is now going to change a bit. This report will focus on some major emulation improvements and it will explain what these entail in general. Thereafter a few select more interesting games and how they were improved will be covered. Every improved game will not be covered because there were simply too many, and evaluation of earlier reports indicate that it isn’t interesting content either.

First of all are the compatibility database statistics for the month of July. Take note that the last database update was performed a day before the major emulation improvement known as “LLE gcm” was merged, meaning the hundreds of games improved from this are not listed in the figures below, or even on the compatibility database yet.

Table of Contents

Improvements
Games
Commits
Upcoming
Conclusion

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (June 2017)

Looking at the GitHub statistics, 18 authors have pushed 201 commits to the master branch. Here 257 files have changed and there have been 14,559 additions and 5,088 deletions of lines of code. What improvements came from these changes? Let’s take a look:

Continue reading Progress Report: July 2017

RPCS3 AppImages are now available for Linux!

Nearly six months after the Patreon launched the RPCS3 team have finally improved RPCS3 on Linux to the point where it has reached compatibility and stability parity with Windows. Thanks to the hard work by hcorion we can finally start to provide pre-compiled binaries in the form of AppImages for easy installation on your favorite distribution.

What Was the Issue?

There were a lot of problems. Back in January, quite literally nothing was working. RPCS3 would crash instantly upon booting any game, if the program itself would even start at all. Moreover, additional functionality like the debugger, framerate counter, and firmware installer were completely broken too. While many of these auxiliary issues were quickly identified and fixed, the fact of the matter was that almost every game would hang after running for a few seconds. This turned out to be much more difficult to fix. This was caused by several different bugs in thread synchronization which were fixed continuously in the past few months. Finally, one last relatively small commit in early April fixed the last bug and suddenly RPCS3 on Linux went from basically nothing to Demon’s Souls and [redacted]. Or so we though, but we quickly found out that the LLVM recompiler was completely broken for a lot of users who just got completely nonsense errors. We encountered strange and esoteric bugs and oddities about LLVM and how various Linux software, including the Mesa drivers, were using it. These problems made RPCS3 unusable for a lot of people. A lot of false flags and red herrings later the bugs were fixed not by changing any code, but by using rare compiler flags. RPCS3 on Linux is now working as intended for everyone, including AMD and Intel graphics users with modern Mesa. Even Vulkan with Mesa is now working!

PlayStation 3 Games on Linux

Below are some popular PlayStation 3 games showcased running on Linux. Performance is about the same as on Windows, perhaps even a few percent better in some very intensive games like [redacted]. But take note: These images were either captured on a laptop with a very old i7-2670qm CPU, or a fast desktop with a i7-4770 CPU.

Continue reading RPCS3 AppImages are now available for Linux!

Progress Report: June 2017

Another month another colossal RPCS3 progress record. But first things first, take a look at this beautiful introduction to RPCS3 video that reznoire made:

[redacted]

Moving on to some statistics the numbers alone this month are quite impressive, and that is not even counting games that improved from emulation improvements that are still work in progress.
Game Compatibility: Game Status

Game Compatibility: Monthly Improvements (June 2017)

Looking at the GitHub statistics 16 authors have pushed 140 commits to the master branch. Here 353 files have changed and there have been 19,729 additions and 17,660 deletions. These numbers are much larger than normal, and several exciting changes are behind it. This month RPCS3 transitioned away from the previous GUI toolkit Wx to Qt which in turn also lead to several user facing interface improvements. This separate report goes into significantly greater detail regarding what work has already been done on the GUI, and what work is planned to be done in the future. On a lower level, Nekotekina reworked the entire PPU LLVM recompiler to greatly increase compatibility of it. kd-11 fixed various graphical issues affecting hundreds of games, most notably broken shadows and depth of field in various titles. Moreover kd-11 also enabled Vulkan on Linux and pushed support for it to the master branch. Nekotekina and Numan also worked on .sprx relocations in general and in the LLVM recompiler which in simple terms is a compatibility and performance improvement.

The talented developer Jarves is working on a very important core improvement. It is not yet finished or merged to the master branch, but as work on this has been going on in public, and a lot of people have already tested it and submitted results this report will cover the work anyway. Jarves is working on “LLE gcm” which is huge and here is why:

LLE stands for Low Level Emulation. In RPCS3 this means that a PS3 operating system module is run directly as it is via lower level emulation methods. This is great for compatibility because it doesn’t involve much guesswork and is very accurate. As far as the games are concerned, they are working with the exact same operating system methods with the exact same implementations here as on a real PS3.

gcm or “graphics command management” is the part of the PS3 operating system responsible for creating and managing various graphics commands, including everything from how to set up vsync to graphics memory allocation.

LLE gcm is huge. A very core part of emulating games is now done exactly the same as on on a real PS3, and the result is amazing. For example Red Dead Redemption no longer hangs almost instantly. The Last of Us doesn’t crash immediately and actually shows a loading screen and goes through a lot of initialisation successfully. [redacted] no longer randomly crashes. Various high end games that did nothing before are now starting to boot, for example the Yakuza series where Yakuza 3 went from nothing to ingame, and Yakuza 4 and 5 went from nothing to loading screens. One could go on and on about what a substantial compatibility improvement LLE gcm is (and I will). Now combine this with the greatly improved LLVM recompiler and over 40 commits of graphics fixes and you have a month of quite insane progress. This would be a great segway into looking at improved games and such, but Jarves did one other important contribution that deserves its own section.

Continue reading Progress Report: June 2017