Progress Report: March 2018

March has been an amazing month full of progress for RPCS3. This month marks the first time that games in the “Nothing” category fell beneath 1%! We had massive strides in graphical accuracy, along with improvements in SPU performance and game compatiblity. From giant pull requests that generated tons of hype, to a small typo fix that had unexpectedly wide-ranging impacts, this month has had it all.

Table of Contents

Major Improvements
Games
Commits

Below are the usual compatibility database statistics, with all metrics moving in the right direction. The tiny number of games left in the Nothing category is quite exciting, especially considering this was the largest category not long ago.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (March 2018)

Major RPCS3 Improvements

Removal of the MFC Thread (#4335)

In the last few hours of the month, Nekotekina made accuracy improvements to RPCS3’s SPU emulation. Since SPU MFC cannot really be explained easily, here is the short version:

The processor in the PlayStation 3 is called Cell and can, in simple terms, run 8 threads concurrently. Cell has a single primary CPU core with simultaneous multithreading (marketed by Intel as “Hyper-Threading”) called the Power Processor Element (PPE), meaning it can run two threads concurrently. The Cell also has six Synergistic Processing Elements (SPE), which each consists of one Synergistic Processing Unit (SPU) and a Memory Flow Controller (MFC). While the PPE and the SPE’s are all on the same package so to speak, the SPU cores cannot access the main memory directly but must instead go through the MFC.

The old and now replaced implementation had a separate MFC thread running concurrently with the SPU threads. This was implemented as it was assumed it would perform faster. However, in practice, the heavy handed use of thread synchronisation and other factors greatly diminished any theoretical gains. Moreover, it never quite worked right, causing all kinds of thread synchronisation issues with subsequent bugs and crashes.

Now, Nekotekina has removed the spare MFC thread. As a result, memory access emulation is instead done within the SPU threads. While being not only safer and stable, it also does not seem to negatively affect performance at all. Case in point– some games with known thread synchronisation problems are now running even better! Everything from the obscure Ar Tonelico Qoga (much less likely to crash/hang on loading screens and the world map) to even The Last of Us (still crashes yet less often, and, with a lot of patience or luck, it is even possible to go Ingame!).

Essentially, this is mostly a stability improvement. Games that used to hang or crash could be improved by these changes.

Typo Fix (#4311)

RPCS3 has had quite a bit of development towards supporting 3D textures, but due to this typo, it was never truly enabled. Changing a single ‘2’ to a ‘3’ ended up improving issues in multiple titles, including Skate 2, LittleBigPlanet Karting, Dante’s Inferno, Infamous Festival of Blood, and numerous others. Credit to pauls-gh for finding the issue.

Major RSX Improvements (#4265)

The first kd-11’s huge PR of the month brought included optimizations, removal of some hacks (which may cause some regressions for a while), and numerous accuracy improvements. For a full list of exactly what changed, check out the PR itself. Notable improvements from the PR are fixed shadows in many games, including Red Dead Redemption, Uncharted and Skate 3. This also got Star Wars: The Force Unleashed – Ultimate Sith Edition to show something for the first time, though it still crashes shortly after booting. Further details for kd-11’s work this month can be found on his Patreon report.

The following video showcases improvements from the two RSX changes previously mentioned:

Epic RSX Fixes (#4312)

The second kd-11’s huge PR of the month brought improvements to hundreds of games, mainly those using Unreal Engine. This changeset was initially submitted as a minor change to help with some bugs in Unreal Engine powered titles. It had been noted the brightness on these games was way too high, shadows were missing and bloom covered everything in blinding light.

It took a very long debugging session and comparison with PC ports to discover that assets in the engine are treated as gamma-corrected (sRGB colorspace) vs the regular linear RGB colorspace RPCS3 used for all its textures. This process involved sampling pixels over many draw calls and the discovery that on RPCS3, simply reading the texels was giving wrong results. The parts required to detect this seemed to have been implemented a long time ago so a minor fix was submitted to detect sRGB encoded textures when rendering. This fixed the brightness in unreal games but broke many other titles making them extremely dark.

It was at this time that kd-11 noticed some strange shaders in these games, often fragment shaders with only one instruction were read as a texture input and wrote to the framebuffer. That makes no sense as a 2d copy engine exists and the games were already using it, so why did they do this? On a hunch that something similar to the OpenGL extension framebuffer_srgb, kd-11 added some code to change the output colors from these seemingly useless shaders and the brightness was seemingly fixed. This was further supported by the fact that in subsequent draws using previous framebuffers as inputs, the framebuffers had their sRGB encoding flags set despite their contents being rasterized by the GPU. This was a clue that a lot more needed to be done since the initial assumption was that only some sRGB texture formats could be used this way. A register controlling gamma corrected framebuffer writes was found (id 1fec) and gamma correct output was implemented.

More command dumps showed other unexpected formats being sent to the GPU with the sRGB encoding set meaning simple hacks would not suffice. Texturing opcodes were rewritten to emulate some more bits controlling behavior of texel reads and the regressions got fixed without breaking the unreal titles. There are a few other changes in the set affecting other parts of the emulator, but the gamma correction issue is what most users will immediately notice. Further details for kd-11’s work this month can be found on his Patreon report.

A video showcasing this changeset was published on our channel, have a look at it if you haven’t already:

Implementation of vertex_data_base_index (#4356)

Use of the ‘vertex_data_base_index’ register was implemented by jarveson, taking into account the base index when creating/using index buffers. Fixes graphics in several games such as Knights Contract, Valkyria Chronicles, LEGO Star Wars III: The Clone Wars, Castle of Illusion Starring Mickey Mouse, RIO, Alone in the Dark, Blood Drive, amongst several others.

Games

This month’s games section will focus mostly on games that were not showcased on our highlights video. We had a huge amount of reports of improved games this month, so unfortunately we can’t show them all. Below are some of the interesting games that improved in one way or another.

The Last of Us

Perhaps the most exciting improvement this month was The Last of Us. The game that has often been called “the Holy Grail of emulation” is finally showing significant signs of life in RPCS3. While it is not currently in a Playable state, we are now closer than ever before! Thanks to improvements in stability from MFC changes brought up by Nekotekina and eladash aka elad335, The Last of Us can now consistently reach the main menu and display some graphics. We’ve got a ways to go before we can call this title Playable, but this is truly a milestone for RPCS3.

With a lot of patience it is possible to get past the cascade of race conditions to even see some gameplay at a glorious 1 fps:

Skate 3

As shown in the video below, Skate 3 is now running pretty well! With fixed shadows and generally improved performance and stability, Skate 3 is running better than ever in RPCS3. Running at 4K resolution, you can see the graphics clearly have held up surprisingly well over the years.

Crysis

But can it run Crysis?
That’s right, Crysis 1 and 2 are now Ingame in RPCS3 thanks to eladash! The PS3 version of Crysis was an interesting one. While it certainly doesn’t have quite the same texture detail or resolution as the PC original, the PS3 version runs on CryEngine 3 instead of CryEngine 2. This game isn’t anywhere near Playable yet, since it suffers from graphical issues and random crashes, but just a few weeks ago this game didn’t do anything at all.

Supersonic Acrobatic Rocket-Powered Battle Cars

Several graphical issues were fixed on SARPBC, which now renders correctly.

inFamous: Festival of Blood

This well known title also improved greatly throughout the month.

Drakengard 3

This Unreal Engine 3 game received major improvements in rendering thanks to kd-11 and shortly afterwards an issue where the game would crash when checking for trophies has also been fixed thanks to Megamouse. A user in our Discord channel even managed to finish the game with good performance and audio.

Asura’s Wrath

Illynir, another RPCS3 user, managed to complete Asura’s Wrath including DLCs without any major issues and good performance. This game made it into the Playable category this month!

Below is a general gameplay video:

Ninja Gaiden Sigma

This title has always had good performance, but only recently received fixes to audio and graphics thanks to Numan and kd-11. Previously, audio wouldn’t work at all and the colors were quite broken due to an issue with how rpcs3 rendered the Ingame fog. After both of these issues were fixed, Ninja Gaiden Sigma has become Playable!

Here is a video showing how the game improved:

LEGO Batman 2

This is one example of the numerous games fixed by jarveson’s index_base_offset implementation.

The following games have been improved by the Epic RSX Fixes changeset, mostly improving in a similar way – fixed shadows and gamma:

NIER

Journey

Gran Turismo HD Concept

Tony Hawk’s Pro Skater HD

Batman: Arkham City

Destroy All Humans! Path of the Furon

Life is Strange

Commits

As always, this is not a complete list of PRs or commits, nor does it necessarily list every single thing done by a given PR. For a full list, see PRs merged in March. Many of the unmentioned PRs are simply small updates, fixes, and other small improvements, sometimes only to the GUI.

Nekotekina

4268 – Introduces some changes to the way RPCS3 handles Blu-Ray mounting, which allows some lazy PSN ports which present themselves as disc games to boot. Among others, this takes the PSN version of Demon’s Souls into Loadable category, and steadily decreases the count of games in Nothing;

4276 – Fixes a few cellGame regressions introduced by the previous PR (4268);

4294 – Locks the log file while it is being used by the emulator. This prevents incomplete and corrupted log files from being shared on forums or Discord, which makes it easier to provide tech support;

4299 – This usability improvement allows you to pre-compile the PS3 firmware and create LLVM cache for games before actually starting them, which means you can do it once for all your games easily instead of waiting for the compilation to finish for every game you start. But most important of all: This can pre compile every .sprx module that a game might ship with.This makes the PS3 version of Okami and Final Fantasy X among others much more Playable. For example, Final Fantasy X has a sprx for every single spell effect in the game, which caused a lot of compiling to occur;

4335 – Gets rid of MFC Thread. Read more about it at the beginning of this report.

kd-11

4265, 4296 – RSX fixes, read more about it above;

4312 – Significantly improves lightning, colors and addresses other graphical problems in a whole lot of games, including Drakengard 3, NieR, Yakuza 3/4, Uncharted, Journey, Red Dead Redemption and many others. Every game that is powered by Unreal Engine 3 seems to be affected by this;

4337 – Handles negative fog distances, which should result in graphical improvements in games that make use of those;

4344 – Fixes driver crashes that RPCS3 users may have experienced when using older AMD GPUs with the Vulkan renderer;

4354 – Fixes a couple regressions with previous changes resulting in incorrect lightning and an issue with macro code.

jarveson

4241 – Takes several games from Nothing to Ingame, including the PSN version of Ratchet & Clank: Quest for Booty, as well as demos for The Chronicles of Narnia: Prince Caspian, LEGO Batman: The Videogame and Indiana Jones: The Original Adventures;

4253 – Adds vertex data base to offset and masks before translating RSX addresses. Fixes WET and helps with fixing Crysis, also being a step forward to fixing Ratchet & Clank games that do not work yet;

4261 – Fixes alpha test on VK/GL by moving it outside of fp_main. This enforces alpha test is still done even if shader asm has early return. Fixes Kingdom Hearts 2 Timeless River World transparency;

4252 – Fixes raw spu interrupt thread overflowing: The stack wasn’t getting reset between interrupt calls with raw SPUs, leading to a stack overflow in some cases. Makes Vancouver boot, now gets to main menu if not further;

4356 – Implements index_base_offset which improves graphics in many games, including Valkyria Chronicles, Yakuza 3, Fracture and Middle-Earth: Shadow of Mordor between many others.

Megamouse

4240 – Enable background images for floating dock widgets, for themes usage;

4281 – Allows hiding of specific entries in the game list and adds an option to disable the distracting shader pre-compilation text pop-ups in the game window of the emulator;

4322 – Immitates some PSN functionality required by Luftrausers to load, taking it Ingame;

4321 – A minor UI usability improvement which allows the user to configure game icon sizes for Grid and List modes independently;

4353 – Fixes a minor UI bug which caused game icons not to display in the game list, and improves user interface code in general;

4316 – Improves support for PSN trophies in games, which, among others, fixes some crashes in Killer is Dead, Drakengard 3, and Lollipop Chainsaw;

4349 – Improves current HLE cellGem implementation;

4355 – RSX Debugger layout fixes.

hcorion

4254 – Updates the Vulkan submodule, which fixes building of RPCS3 binaries.

AniLeo

4249 – Usability improvements for the YoRHa theme;

4361 – Removes an unnecessary Qt extension from the Windows packaged builds, decreasing the archive size.

VelocityRa

4083 – Introduces some support for emulating the PS Move. While this doesn’t add support for the actual controller itself, it allows you to map Move buttons to gamepad ones, and allows many of the games that require Move to boot and some to go Ingame;

4285 – Implements a deprecated function that old Playstation Move/Eye games require. Examples are ‘EyeCreate’ and ‘The Eye of Judgement’ (first released Move game);

4278 – Improves the LLVM recompiler fixing, among others, crashes in ‘Portal 2’ and ‘Castlevania: Harmony of Despair’.

elad335

4256 – Introduces some UI usability improvements. In particular, the log display is now in Stack mode by default, which means that repeats of the same errors or other messages won’t spam the log window repeatedly;

4331 – Fixes some SPU emulation issues, taking Crysis 1 and Far Cry 4 to Intro, Crysis 2, F1 Race Stars, Toybox Turbos and Prototype 2 to Ingame, Harry Potter and the Deathly Hallows: Part 1 from Nothing to Loadable. It also fixed issues with Def Jam Icon and Baja: Edge of Control.

TGEnigma

4026 – Improves the debugging functionality of RPCS3, by adding a “Step Over” function, new keyboard shortcuts and changing the “Go To Address” function.

scribam

4251 – Fixes the way the PS Camera imitation is handled. This takes Rapala Pro Bass Fishing from Nothing straight into Ingame;

4308 – Improves performance of some AMD GPUs for Linux users with the open-source Mesa drivers;

4292 – Moves the USE_NATIVE_INSTRUCTIONS variable from the Travis config to CMake, resulting in RPCS3 builds compiled manually from source on Linux closer to the official AppImage binaries improving their performance;

4314 – Improves the way RPCS3 handles invalid/empty paths passed to file-handling functions, taking E.X. Troopers out of Nothing and into Ingame;

4329 – Makes it possible to build RPCS3 on OS X. There’s still a variety of problems in the build actually being anywhere near useful, though. In particular, LLVM support doesn’t build yet. Moreover, there are many known issues with Vulkan and OpenGL on OS X, which are beyond our own scope of development. In other words, this doesn’t indicate OS X support just yet, in any capacity, just an eventual possibility thereof;

4365 – Stops RPCS3 from crashing on games which use interlaced frames in video files and when the HLE video decoder is used. The LLE video decoder would always have worked though performance would suffer. Among others, takes Stealth Inc: A Clone in the Dark from Ingame straight to being Playable and moves Truck Racer Ingame when the HLE decoder is used.

pauls-gh

4320 – Remove Volume Texture Compression (VTC) tiling for Vulkan, DX12 and ATI (OpenGL). Fixes rendering issues with compressed 3d textures (e.g. fonts on Tales of Vesperia/Xillia);

4350 – Moved old surface copy before texture upload. Resultes in Tales of Vesperia no longer requiring Strict Rendering Mode allowing users to use Resolution Scaling.

SelfGamer

4325 – Allows RPCS3 to detect whether the user’s operating system supports specific AVX instruction sets, which prevents crashes in a situation where the CPU itself supports these instructions but the OS does not.

kevinxucs

4324 – Fixes git-version.h not correctly updating.

GeniusMage

4311 – Changing just a single character in the code fixed visual issues with Tales of Vesperia, improved Skate 2, LBP Karting and some other games.

danilaml

4279 – Fixes a Qt-related environment variable in the AppVeyor configuration, which was necessary to fix building RPCS3 for Windows.

jbeich

4277 – Fixes issues that made it impossible to build RPCS3 on *BSD operating systems, further expanding cross-platform support.

Nicba1010

4269 & 4273 – Fixes Travis, which is the build system we use to generate Linux AppImages for RPCS3.

zekesonxx

4257 – Stubbed sys_crash_dump get/set user_log_area which allowed Destiny to boot properly and display Intro graphics.

jjsat

3918 – Addresses issues with the way some games handle saving and loading data. This improves Grand Theft Auto IV, Tomb Raider, Tekken, and fixes the save-related errors in demos for Alien Breed: Impact and Alone in the Dark, among others.

Asinin3

4345 – Indicate speed differences between Interpreters and Recompilers for less confusion.

Kravickas, Talkashie

4333, 4328 & 4250 – Fixes for some typos and odd phrasing in logs and in the user interface, with no effect on game compatibility.

Closing Words

If you like in-depth technical reports, early access to information, or you simply want to contribute, consider becoming a patron! All donations are greatly appreciated. RPCS3 now has two full-time coders that greatly benefit from the continued support of over 700 generous patrons.

Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by ssshadow, Ani, Asinine, Talkashie, kd-11, 13xforever and Yahfz.