Welcome to our (2018) December Progress Report! We are extremely happy to finally push this report out to you guys after the many delays, for which we tried to make up with other kinds of updates along the way. We hope that you’ll enjoy our deep dive about the last month of the past year, and find its stories equally as intriguing.
December has been an absolute whirlwind for RPCS3 with improvements made to various parts of the emulator. Nekotekina greatly improved performance with the introduction of Approximate xfloat while kd-11 fixed a wide variety of graphical issues affecting multiple AAA titles. Apart from our full-time developers, our regular contributors also made a plethora of improvements to the texture cache predictor, cellCamera emulation, controller/mouse support, GUI and tons of bug fixes. We’ve tried to detail as many as we possibly could in this report. So without further ado, let get into it!
In addition to the following report, further details about Nekotekina and kd-11’s work during December and their upcoming contributions can be found in their weekly reports on Patreon. December’s Patreon reports were:
Table of Contents
This month saw improvements to various aspects of the emulator and the compatibility list is no exception. At the start of 2018, the compatibility list was revamped to allow us to merge multiple game IDs for the same game into one single entry. This helped reduce duplication of entries and enhance the quality of the statistics we provide. While we were able to merge IDs of different regions together, due to the way the compatibility list database was structured, titles with multiple IDs from the same region could not be merged and hence were shown as separate entries. Not wanting to leave the job half done, AniLeo refactored the compatibility list once again this month to now allow us to merge IDs from the same region as well!
With this change, we can eliminate unnecessary duplication entirely and significantly improve the quality of our statistics to properly represent the amazing progress made with the emulator. If you take a look at the metrics for this month, you will notice that the number of games has decreased across all categories. This is solely due to the above mentioned revamp of the compatibility list. However, if you take a look at the compatibility history page, we can see that the Playable category has increased by over 15 titles and Ingame category has increase by over 30 new titles. The nothing category also reduced by one, bringing the total to just 3 titles!
On Git statistics, there have been 4,802 lines of code added and 2,530 removed through 105 commits by 18 authors.
Major RPCS3 Improvements
GT Fixes (#5436)
In December, kd-11 fixed numerous issues in a large variety of games. For starters, he laid the foundation for fixing display output issues, where rendering was squeezed into one corner of the screen. Shader decompilation was also improved in several ways and the fixes that stood out the most were correcting how double assignments (like x = y = z) are resolved, the reimplementation/ refinement of conditional execution so as to avoid data corruption, and the separation of depth and stencil data for shader inputs, were they to be accessed simultaneously. While the first of the three is pretty self-explanatory, the latter two need some further explanation, so let’s do precisely that!
First, let’s see what conditional executions needed a fix for! Say you had three pixels (a, b and c), and you wanted to modify ‘a’ pixel’s color so that a = b * c (in color values). Since a pixel’s color is determined by the values of their channels (alpha, red, green, blue) this means doing this multiplication for every channel, one-by-one (so, not following the official notation, a.red = b.red * c.red, a.blue = b.blue * c.blue and so on). You, being unhappy with this result, decided then that you wanted, say, the green channel excluded from this process. You can do that by relying on a so-called choice condition register, in the following fashion: you put a pixel value in it, and you set each of its channels’ value to 0 or 1. If a channel is 0 in the pixel stored inside the choice condition register, it means that exact channel will be excluded from the process laid out above. Knowing this, you set the green channel’s value to 0, set 1 for all the others, and you go ahead with the execution. You can now check for each channel’s value in the register before executing the original command for that given channel (so before you do a.green = b.green * c.green you test against cc.green, where cc is the choice condition register).
This is nice and all, but let’s say you wanted to go even further than just multiplication. In a more abstract sense, this means putting the ‘a’ pixel into some function, that function does something with it, and you read the resulting value back to ‘a’ for each channel. Anyone who ever did software development will now raise their heads up instantly: since we are assigning the result of function(a) back to ‘a’ channel-by-channel, and not at once, we repeat the function again and again, but use the channel values that have already been altered with each new run instead of the original ones. In hindsight, this is an obvious data hazard and was easily fixed by moving the function execution before the assignment, instead of repeating it for every channel. Of course on GPUs, this is quite a bit more elegant: a = lerp(a, function(a), cc);. This change affected games where corrupted color was perceptible:
Now that we got this one down, let’s see what the separation of depth and stencil data was needed for. Due to API limitations, on PC, you cannot read depth and stencil values simultaneously from within the same shader. On the PS3 however, this is not a problem: you can do so by casting (~forcing) the given D24S8 texture (depth data of 24 bits + stencil data of 8 bits = 32 bits) into an ARGB8 texture (alpha, red, green, blue, with each being 8 bits, so in total = 32 bits), and then reconstruct the original depth and stencil values inside the shader, using the individual channels.
In order to retain the original form of the shaders, we previously tried to mimic this process as follows: we casted the depth data into color bits (R,G,B), and stubbed the stencil value into being 1 always, residing in the transparency channel (A). This worked surprisingly well for the longest time, but as it turns out, games like God of War or titles from the Gran Turismo series used the stencil part to encode data required for proper font rendering.
To fix this, kd-11 gave up on retaining the original shader’s form and now passes the depth and stencil data as separate textures. The only caveat is, that instead of having 16 inputs at max, shaders can now have up to 32 inputs when all the input textures are needed to be split – something that isn’t possible on all drivers. In turn, proper font rendering (among others) is now possible. This improvement benefitted a multitude of titles including titles from the Gran Turismo series that saw a distinct uplift to graphical output:
Approximate xfloat (#5430)
Back in August 2018, Nekotekina implemented accurate handling of extended floats when using the SPU LLVM recompiler. As previously detailed in that month’s progress report, this brought massive improvements to accuracy but with a significant decrease in performance. Due to the negative impact in performance, Accurate xfloat was retained as a toggleable option that was disabled by default. A large number of titles compulsorily required Accurate xfloat to be enabled when using the SPU LLVM recompiler as they would otherwise suffer from broken physics, missing geometry, incorrect models, etc.
While Accurate xfloat provided the most effective solution, its performance hit was significant enough that only high-end CPUs could utilise the option and still maintain playable performance. Users were left with the choice to either upgrade their CPU to enjoy full accuracy or stick with an unplayable experience. There was a clear need to have a midway solution that allowed for better accuracy without degrading performance. This was also clearly evident as multiple hacky builds of RPCS3 began to popup that simply carried over the quick workaround that kd-11 had previously implemented to the SPU ASMJIT recompiler.
To fill this void, Nekotekina implemented Approximate xfloat which effectively adapts kd-11’s workaround for certain instructions while forcing accurate handling for others. This resulted in huge performance gains in games that previously required Accurate xfloat such as God of War 3, Ninja Gaiden 2, Sly Cooper Thieves in Time, Ni No Kuni, Uncharted, Macross, Wipeout HD, Heavy Rain, Dante’s Inferno and Journey. Using this options allows users to reap most of the benefits from Accurate xfloat without any performance reduction. Consequently, this option is now enabled by default when using the SPU LLVM recompiler.
With the implementation of Approximate xfloat, the SPU LLVM recompiler now boasts higher accuracy and performance compared to the SPU ASMJIT recompiler by default. With further optimisations and fixes, the SPU LLVM recompiler is set to become the definitive backend to use in RPCS3!
Texture Cache Predictor Rewrite (#5276)
In our September report, we covered ruipin’s various texture cache related changes in great detail, but most importantly, we also introduced the concepts of texture locking and hard faults. Since the aforementioned hard faults are a key concept of this change, here’s a quick refresher: when the Cell tries to read from a read-protected texture region, we first have to make sure that said region is up to date. If that isn’t true, we need to halt emulation and flush the texture from VRAM to RAM to resynchronise – and that is what’s called a hard fault. Its downside is sadly obvious: if we need to halt the emulation process to wait for such texture uploads, performance will suffer, and suffer greatly.
To alleviate this, there was already a predictor algorithm in place. As its name suggests, it’s job was to predict if a game will need a certain texture ready, so that by the time the game actually asks for it, we’ll already have it uploaded into the RAM. Unfortunately however, since predictor data before the rewrite was stored in each texture’s data structures, the heavy reuse and reset of those structures cause by the rewrite made the predictor plenty less reliable. Adding insult to injury, the original predictor wasn’t terribly sophisticated to begin with, so with September’s changes in place, its effectiveness by and large reached a minimum.
This is where ruipin’s predictor rewrite came into play. To restore the predictor’s hit rate, he redesigned its heuristic from scratch. Textures are now kept track of in a dictionary via so-called “predictor keys”, which are basically hashes of batched-together texture metadata of each texture in the cache. This enables the predictor to now correctly react whenever it comes across a previously seen texture, and know more about its usage statistics. In addition to this, various fine-tuned adjustments were made for edge cases, such as overly frequent flushings of a given texture, and more. This change not only restored the predictor’s efficiency, but even increased it beyond that of the original, leading to way less hard faults in games such as TLOU or R&C ToD, and virtually no more mispredictions (pointless flushes).
Every month, our inhouse Quality-of-Life developer Megamouse tries to improve your experience using RPCS3 and December was no exception. The controller handler was restructured to allow the smooth implementation of some much needed features. These improvements benefitted multiple input devices such as the DualShock 4, XInput devices and computer mouse.
The first improvement was the restructuring of the pad implementation to benefit all controller backends. As users of RPCS3 would already know, the emulator does not allow its settings to be changed while a game is running. For any modified settings to take effect, the game would have to be restarted and this was also true for the Pad settings. This soon turned into an inconvenience for many users who had to restart the game simply to adjust deadzones or vibration intensity. Some users also needed to restart the game if their controller accidentally disconnected during gameplay.
To alleviate these issues, the Controller handling was improved to allow for live detection of controller status and application of configurations made in real-time. Users can now change and save all pad settings while being in-game. It is also possible to switch from one controller backend to another seamlessly, for e.g., switching from Xbox One controller to Keyboard+Mouse. Once users have made the necessary changes, they can simply hit save in the Pad configuration dialog and the new settings will be immediately applied to the game. Along with these improvements, the connection status of controllers will now show up in the log so users will be able to clearly see whether or not their controllers were detected by the emulator when connected. In the event that the controller is disconnected, no remapping will be allowed to preserve the saved settings for such controller. Any button reassignment that was being made during such disconnection will also not be saved. This prevents accidental errors when the controller’s connection is weak.
The second feature implemented was improved support for binding Mouse to analog sticks. While this feature was implemented in August 2018, the implementation had some drawbacks. The primary deficiency being that native deadzones offsets weren’t saved and would have to be set every time the game was booted. The second being that the process of modifying the offsets was primary driven through log messages making it that much more to set-up. Fortunately, all of these drawbacks were addressed this month. Mouse parameters set now save along with the Keyboard config profiles so users no longer have to reapply the settings every time. The Pad configuration dialog now also displays neat options to easily modify the mouse acceleration and deadzones thresholds.
Apart from UI improvements, stick interpolation was also added to help further improve the experience when using the Keyboard. When using the Keyboard, input for analog sticks are restricted to only 4 keys (e.g. W A S D) and consequently only 4 directions. However, when using a regular controller, users have the benefit of moving in any direction that the analog stick resolution supports. Therefore, emulation of analog sticks with the Keyboard has an inherent disadvantage that could negatively affect a user’s experience. This is where stick interpolation comes into play. RPCS3 now smoothens the key input for the left and right analog sticks over time using a basic lerp (linear interpolation) per stick axis (4 axis in total, 2 for the left and 2 for the right).
By default the interpolation level is set at 1.00 (i.e. no lerp is applied). Lowering this value will make the analog stick smoother to move. This is especially useful when, for example, you want the character to turn normally instead of simply snapping into that direction. However, do remember that setting the interpolation value to 0.00 will result in no movement to be recognised from that analog stick.
Apart from these awesome updates, there were also some miscellaneous improvements made to other specific controller backends. Settings for the DualShock 4 was made more dynamic by using an index from 1-7 instead of HID-Device-IDs. In line with the above changes, the DualShock 4 backend was also improved to allow for seamlessly disconnecting and reconnecting during gameplay. For XInput controllers, support for the usage of a single controller for multiple players was also added. While this is a niche feature, users who requested for the same now have reason to celebrate!
God of War 3
God of War 3 is one of the many titles that previously required Accurate xfloat enabled to fix physics issues when using SPU LLVM. Now, thanks to Nekotekina’s addition of Approximate xfloat, this title has seen a massive improvement to performance. Check out the video below to see just how much performance has improved:
El Shaddai: Ascension of the Metatron
Previously, this console exclusive title suffered from random crashes during gameplay that prevented it from being playable. Thanks to the recent stability improvements made to RPCS3, this title is now fully playable! A user from our forums (aptly named Testing) has finished this game on RPCS3 without any major issues.
This month the action packed console exclusive made its way to Playable! With the improvements made to the emulator, the graphical issues and crashes that were previously plaguing this title have now been fixed. Do remember to enable Strict Rendering Mode to fix background flickering.
Motorstorm 3D Rift
This month, the PlayStation 3 exclusive Motorstorm 3D Rift was found to be ingame. Previously this title would crash after the menus. With this, all titles from the MotorStorm series reach ingame! While this game does have decent performance, it does suffer from few graphical glitches keeping it from being Playable.
Thanks to the implementation of DECR console emulation support by elad335, ModNation Racers now goes ingame! However, the game suffers from low performance and graphical issues preventing it from being properly playable.
Golden Axe: Beast Rider
Golden Axe: Beast Rider is yet another console exclusive that was found to progress ingame this month. Within a week of this discovery, user Bnd_ps3 from our forums managed to complete the game on RPCS3. However, from his experience, the game suffers still suffers from random freezes and graphical issues which prevent it from being categorised as Playable. Here’s some gameplay footage to see how this game currently performs:
Thanks to Megamouse fixing a faulty check that lead to inconsistencies in cellCamera, various titles that required the PlayStation Eye now progress further. Operation Creature Feature and The Trials of Topoq managed to reach intros whereas Aqua Vita went ingame! However, proper PlayStation Eye emulation would be necessary for these games to be categorised as Playable.
Everybody’s Golf 6 (Hot Shots Golf: World Invitational)
If you’re a sports fan and consider golf a sport, then we’ve got some great news for you! The latest outing from the Everybody’s Golf series to grace the PlayStation 3 now goes ingame. While the game looks visually gorgeous, performance is too low to consider this title as Playable.
NCAA March Madness 08
If you don’t consider golf a sport, worry not, this saw the classic NCAA March Madness 08 progress ingame for the first time. However, performance during gameplay is low even on high-end hardware. Further optimisations to the emulators is necessary to obtain Playable performance from this title.
Natsuiro High School: Seishun Hakusho
With the multiple accuracy improvements made to the emulator in the recent months, multiple games moved from Loadable directly into Ingame. One such title is Natsuiro High School: Seishun Hakusho. While the game generally has good performance and graphics, certain areas reduce the FPS signficantly, keeping this title from being Playable.
Mobile Suit Gundam: Side Stories
For all mecha anime fans out there, this month saw two such titles go ingame for the first time this month. The first is Mobile Suit Gundam: Side Stories which is a remastered collection of side stories from older generation titles. While this title is relatively stable, it does suffer from low performance and a few minor graphical issues.
Rinne no Lagrange: Kamogawa Dream Match
The second mecha title was the PS3 exclusive title Rinne no Lagrange which was found to be playable this month!
There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details, as well as the code changes themselves. To see the complete list of pull requests directly on GitHub, click here.
5400 – Minor improvement to exception handling in the RSX Thread by detaching VBlank and RSX Decompiler threads;
5414 – Minor improvements to cellSaveData to address the “Already exists” error;
5436 – Fixed display output issues, shader decompilation issues, correction of how double assignments are resolved, reimplementation of conditional execution and separation of depth and stencil data for shader inputs. See coverage in major improvements here.
5393 – Fixed a minor inconsistency in the GUI that caused the “Play” button to display “Pause”;
5412 – Added the ability to customize the title of games present in the game list. The new name will be used for any search and sorting operation and can therefore be used to improve the game list management and customization. Also fixed an issue where you could mistakenly rename titles in the game grid;
5425 – Enabled the usage of one XInput controller for multiple players. Though the functionality was present in the emulator, this feature was not enabled or reported. See coverage in major improvements here;
5426 – Added the ability to make changes to the pad settings during gameplay. Users no longer have to restart the game for the changes to apply and the new changes will take effect spontaneously. See coverage in major improvements here;
5439 – Fixed a faulty check that lead to inconsistencies in cellCamera. This fixed issues with various games such as Aquatopia, Trials of Topoq and Operation Creature Feature and allowed them to progress further!
4908 – Correctly report game window sizes to the graphics backend with HiDPI screens. Windows users should use Qt’s HiDPI mode instead of Windows’ HiDPI mode. Please note that both modes aren’t perfect so this change might be reversed or left up to the user in the future;
5443 – Improved all controller backends by changing the index to 1-n instead of 0-n, adding connection status to the device names, disabling remapping when a controller is disconnected and immediately cancelling current remapping in case the controller is disconnected. Also, specific improvements were made to the DS4 handler by using an index from 1-7 instead of HID-Device-IDs which makes the DS4 settings more dynamic and made it possible to reconnect DS4 controllers. See coverage in major improvements here;
5464 – Fixed unreadable out-of-focus titles for selected games in the game grid by adjusting the concerned style sheets;
5474 – Fixed a mismatch of table header states and icon sizes that occured when the emulator crashed and therefore couldn’t save the header widths. This lead to wrong sizes of the first game list column and occurred frequently when stopping execution in Microsoft Visual Studio;
5476 – Updated outdated tooltips in the settings dialog, updated some setting names to make them more transparent to users, removed “Force Disable Anisotropic Filtering” from the UI dropdown to prevent incorrect usage of the option and moved the obsolete setting “GPU Texture Scaling” to the debug tab;
5468 – Added mouse movement and stick lerp settings that will be applied when choosing the Keyboard as the controller. Also, added the relevant settings to the pad settings dialog for easy modification. See coverage in major improvements here;
5484 – Added a warning log entry “Compiling PPU module: …” at the beginning of a PPU module compilation and a success log entry “Compiled PPU module: …” at the end of a PPU module compilation.
5441 – Added a CMake switch to disable Discord Rich Presence entirely;
5445 – Removed the existing Linux distro whitelist and use the new automated stdc++ checker;
5479 – Deploy Linux binaries to GitHub releases. Now, Linux also has a build history on the website!
5384 – Fixed an issue during the maximization of the game window when using Vulkan with an Nvidia GPU where the render did not resize properly. The fix was to simply send the resize message to RSX on the last possible message;
5388 – Fixed a regression caused by the above PR where the render did not resize properly when using Vulkan with an AMD GPU;
5399 – Fixed a regression in cellSaveData that caused titles like Battle Princess Arcadias to crash on launch. This was fixed by adding more result handling from funcStat CB in savedata_op. This fix actually came from an investigation as to why some games started with no sound and you had to go into settings to increase the sound volume;
5304 – Fixed an error in cellFsGetFreeSize where previously RPCS3 used the full path to determine if free space was available when the PS3 only used the device (/dev_hdd0, etc.) in the path. This fix addressed issues in games like LittleBigPlanet, No More Heroes: Heroes’ Paradise and Moon Diver where the game would fail to create a save file due to supposed lack of space;
5450 – Improves trophy handling accuracy;
5457 – Fixed few issues in cellVideoOutConfigure that allowed Karaoke Revolution to progress further!
5475 – Fixed a race condition where the mutex is destroyed whilst an unlock is happening. The crash occurred due to an object being destroyed by a thread while it was still in use by another thread. This fixed the elusive crash in Demon’s Souls that only affected CPUs with low thread counts (4C4T).
5381 – Fixed few RSX image_in command bugs and implemented support for negative scaling. This fixed crashes or blackscreens in a number of titles such as Surf’s Up, NCAA March Madness 08, Mamorukun Curse and Demolition Inc. Games fixed by this fix need Write Color Buffer enabled to work correctly. Further work is necessary for this fix to be fully functional without Write Color Buffers. Also, the RSX capture tool was simplified by removing the memory map save in save state and instead opting to simply preallocate all memory;
5392 – Optimized the PUTLLC MFC command execution by removing 128 byte long redundant read and writes from the compiled code;
5363 – Fixed PPU link register (temporary one-layer return address saver when calling functions) by saving the address of the next instruction even if the jump condition failed;
5419 – Fixed a bug that caused input errors in the native UI. This was achieved by initializing the pad handlers before a game actually initializes them itself, just like a real PS3 also has input before the game starts. This fix also made it easier to follow up with the other pad config changes described above;
5404 – Fixed remaining issues and regressions with RSX FIFO emulation, making some games more stable such as Red Dead Redemption and Ni No Kuni. Also improves RSX debugging in the emulator;
5440 – Improved error handling for the “Null” camera option so that certain games can work properly when no camera is selected;
5334 – Fixed SPU group status after all threads exited individually when not joining the SPU group and simplified SPU group status set after termination;
5459 – Fixed an inaccuracy in sys_semaphore_post syscall where an error code should be returned instead of success status when passing the parameter count = 0;
5466 – Fixed the capture of FIFO commands with a memory state and also improved precision of the conditional rendering checking;
5357 – Implemented support for DECR console (debug unit) emulation and corrected the available memory size reported to games. The DECR console has 256MB of additional RAM in comparison to retail consoles. Since memory status reports to games are not fully accurate, using the Debug mode addresses issues with memory limitations while also making some games expose debug functionality such as printing debug TTY messages and displaying debug menus. Using this mode, Pirates Of The Caribbean: At World’s End now reaches Intro!
5276 – Fixed various regressions with the texture cache caused by PR (5115). In addition, it implements a proper texture cache predictor, that attempts to predict when textures will need to be copied from GPU memory to host RAM, and thus can queue up that operation in advance, avoiding a hard fault and improving performance. See coverage in major improvements here.
5386 – Fixed Marvel Pinball by adding the error check SCE_NP_ERROR_ID_NOT_FOUND to sceNpBasic entry functions;
5451 – Fixed an issue with the time counter in Deathmatch Village where the duration would be stuck at 30 minutes.
5382 – Added game list category filters for PS1, PS2 & PSP games;
5411 – Implemented an option to check the cache size (dev_hdd1/cache) and delete the oldest folders until it’s less than a configured threshold. Users can enable this feature and customize the threshold in the System tab;
5448 – Improves detection of Hyper-threading with Intel processors on Windows.
5473 – Fixed building RPCS3 with musl libc.
5467 – Fixed an exception that caused games to crash on FreeBSD.
5437 – Addressed an issue where the trophy data for a few games (e.g., The Evil Within) is not correctly loaded trophy data.
5433 – Added libevdev-dev as requirement for Debian & Ubuntu to prevent user errors.
5379 – Improve path to discord-rpc submodule when building with non-MSVC compilers;
5387 – Updated hidapi submodule to address mingw compilation issues.
5385 – Added ignore = dirty to all submodules to make git operations faster.
5370 – Fixed paths to PARAM.SFO and game icons for C00 games.
5367 – Added support for macOS bundling for binary distribution.
5472 – Fixed typos in a few tooltips.
Are you still here? Great! ‘Cause there’s way more for us to cover!
January was definitely not a boring way to start the year and was packed with a ton of updates. Not only did we finally have a multithreaded SPU cache compilation path added thanks to Nekotekina (much to the relief of our testers), but we also had native OSK (on-screen keyboard) support added as well as a much welcome audio improvement done over the course of the month. And who knows, maybe many of our users have missed it, but they can now also let their voices out in a particular game…
This was it for December, as well as for 2018. We’ve made fantastic strides over the year, and it’s been a great pleasure for us writers to cover it throughout. We as a project and as a team are extremely thankful for every patron and user who has supported us and helped the project in any way, and we hope they’ll continue to do so in 2019.
We hope you liked this report and look forward to the next one! If you would like to contribute to the project, you can do so either by contributing code, helping the community or becoming a patron. RPCS3 has two full-time developers working on it who greatly benefit from the continued support of the many generous patrons. In exchange, patrons also get special support over on our Discord server and get access to early updates directly from our lead developers. If you are interested in supporting us, consider visiting our Patreon page at the link below and becoming a patron, or join our Discord server to learn about other ways of contribution.
2018, logging out.
This report was written by HerrHulaHoop, dio, Asinine, Megamouse, eladash and Digitaldude555.