Progress Report: June 2018

In addition to the following report, further details of Nekotekina and kd-11’s work during June and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2018-06-05)
Status update from Nekotekina (2018-06-12)
Status update from kd-11 (2018-06-20)
Status update from Nekotekina (2018-06-27)

Table of Contents

Major Improvements
Games
Commits
Upcoming
Conclusion

On compatibility database statistics, we can see a big decrease in Loadable games due to several recent changes targeting issues that kept those games from progressing further. The Nothing category also went down by 2 games, with only 6 games remaining in it.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (June 2018)

On Git statistics, there have been 16 605 lines of code added and 10 558 removed through 135 commits by 21 authors.

Major RPCS3 Improvements

PlayStation 1 Classics (#4802 and #4804)

This month RPCS3 gained the ability to run PS1 Classic titles!

The PlayStation 3 has various types of software and hardware emulators for the most of Sony’s previous generation consoles. For the PS1, 3 software emulators were present within the firmware, namely ps1_emu (runs PS1 discs), ps1_netemu (runs PS1 classics available on the PS Store in .pkg format) and ps1_newemu (supposedly an enhanced version of ps1_netemu). However, similar to VSH, these emulators call sys_storage directly instead of using the file access library. Since RPCS3 currently has no implementation of sys_storage, support for these emulators wasn’t possible.

However, while researching VSH functionality for RPCS3, clienthax discovered that ps1_newemu did not require sys_storage implementation which allowed PS1 classics to boot when the firmware libraries loading was set to ‘none’. However, games would still crash shortly after booting as they called on an unimplemented function, sys_io_3733EA3C, to get input data. Further debugging revealed this function to be cellPadGetDataExtra with a few additional checks. Simply returning the same function fixed the crash and allowed the PS1 Classic titles to progress ingame!

Soon after clienthax’s PR was merged into master, Nekotekina made various improvements surrounding PS1 emulation such as allowing these titles to boot from the recents list, fix PPU and SPU cache from generating multiple times when ps1_newemu only requires it to be done once and allow custom per-game settings for these titles.

Thanks to above contributions, RPCS3 gained the ability to run PS1 Classic titles! While RPCS3 only supports PSN versions of them, most (if not all) are playable and run well. Below are some screenshots from various titles:

MSAA rewrite and Depth Scaling fix (#4698)

Right at the beginning of the month, kd-11 began reworking the MSAA related parts of the codebase. This endeavour resulted in a number of games being fixed, particularly games which suffered from an issue where half of the screen was missing or displayed garbage. But how did an anti-aliasing technique have such an effect on the render output? Let’s dig deeper to find out!

MSAA (or Multisample Anti-Aliasing) is a form of spatial anti-aliasing, designed to get rid of outer-edge aliasing, more commonly known as ‘jaggies’. Since it’s widespread popularity in the 2000s, pretty much all mainstream graphics chips have an inbuilt hardware accelerated support for it – with a different implementation vendor-by-vendor. This resulted in a sticky situation for the emulator: since we can’t access MSAA data directly on the user’s card, we have to fake doing it for the sake of game compatibility, and then in reality, either scrape the AA entirely, or switch to full-on SSAA. While the latter sounds more preferable, it isn’t entirely practical just yet. That is why currently the former option was deployed, but it wasn’t without a catch.

There are multiple MSAA configurations that game developers can utilize. For example, they can pick 2xAA (double horizontal res.), 4xAA (double horizontal and vertical res.), and even switch this quickly between render passes, whilst retaining the actual dimensions (e.g. between 2xAA 1280×720 and 1xAA 2560×720, the buffer would be sized 2560×720 regardless). Due to this however, when such an event occured in games like Tales of Vesperia or Killzone 2, it resulted in having the former pass’ buffer being falsely detected as 1280×720, making the contents of it only filling the left of the adjacent pass’ buffer. kd-11 managed to fix this by keeping track of the actual buffer size, so that the buffers are now rescaled in-between these switches, ensuring that the contents are properly aligned and no squishing happens when the game finally decides to scale everything back to the target resolution.

This issue escalated in games like Backbreaker, where the resolve is handled in a feedback loop with the ROP writing to the same memory block that contains the MSAA samples its reading from. In such cases, just keeping track of how to change the buffer’s size isn’t enough. To alleviate this, kd-11 brought in separate states, called read and write states, which ensures that the MSAA resolves correctly and no image-squishing occurs. With this, Backbreaker finally gets its justice:

Depth scaling was also touched upon while making these changes, as stencils are required for MSAA to work properly. However, since stencil writes are not allowed when writing depth with shaders, further trickery was necessary. To work around this issue, kd-11 implemented a solution where stencil data are now “faked” as color textures, so that they can be transferred to the target texture memory without API constraints and driver limitations. This allows MSAA resolution to now work correctly with stencils.

SPU Analyser Improvements (#4750)

Nekotekina merged several improvements to the SPU about halfway through the month, resulting in better performance in games and cleaner compilation. Neko explored the experimental concept of “big blocks” for the ASMJIT recompiler, which can improve performance in games by bundling more code into chunks, meaning less time is spent on code verification. Neko added three separate modes labeled “Safe”, “Mega”, and “Giga”. The Safe mode only compiles small, select chunks of code together. The performance benefit here is much smaller than Mega or Giga, but it has a substantially higher success rate. Mega mode will compile entire functions at once, with Giga mode going a step further by stringing together multiple functions into the same chunk. While Mega and Giga mode are much less stable than the Safe mode, the CPU time required for code verification is drastically reduced, sometimes being orders of magnitude lower.

SPU LLVM code generation was also improved by Nekotekina this month. The groundworks for some major optimizations were set which will allow for cleaner compilation. Once all is said and done, this should result in less garbage code being compiled. For a more detailed description on the above changes, please refer to Nekotekina’s Patreon post here.

Games

Yakuza Kenzan

Thanks to jarveson’s fixes, this beloved AAA PS3 exclusive has gone ingame! Though the graphics looks stunning, the game suffers from low FPS and occasional texture flickering (as shown in the third image).

Dynasty Warriors Gundam

Thanks to Farseer2’s fix for sys_spu_thread_connect_event, this console exclusive went from Loadable straight to Playable!

Tekken Tag Tournament 2

Thanks to the work done by kd-11 in fixing the half-screen bug, this console exclusive is now considered Playable! However, Tekken Tag Tournament 2 still has other lingering issues such as silent intro dialogues and background music while also requiring “Strict Rendering Mode” to be enabled for the Fallen Garden and Heavenly Garden stages to correctly display. Enabling Write Color Buffers also fixes black patches appearing on characters clothing, however there’s still an issue where some of the clothing incorrectly renders as being grey/white.

God of War: Ascension

Another AAA Playstation 3 exclusive from the God of War series now properly goes ingame! However, as with Yakuza Kenzan, this title also suffers from low FPS and various graphical issues.

Final Fantasy XIII, XIII-2, XIII Lightning Returns

This month, all 3 installments of Lightning’s Saga went ingame! The games, however, still suffer from low FPS in some areas and minor graphical glitches which keeps them from being playable.

Sly Cooper and the Thievius Raccoonus

This Playstation exclusive went from Intro straight to playable this month after one of our testers “Asinine” finished the game from start to end on RPCS3! Take a look at the video below to see just how well it’s running!

Sly 2 and 3

Similar to Sly Raccoon, these PlayStation exclusives also saw a major boost in compatibility. Both Sly 2 and 3 now go ingame! Each of these games suffer their own set of issues, Sly 2 will give players a black screen during the prologue sections but can be bypassed using a save. Sly 3 suffers for worse performance and issues with certain levels where there is two cameras. Both Sly 2 and 3 also currently have issues with the autosave feature and require manual saving and make sure to save twice in the same slot or it won’t create save files properly! This is only necessary with Sly 2-3 and another game, “Rune Factory Tides of Destiny”.

Ratchet & Clank series

Thanks to the “Kot fixes” by isJuhn, various Playstation exclusives developed by Insomniac Games now progress further. Among them are R&C Future: A Crack in Time, R&C Into The Nexus, R&C: All 4 One and R&C: QForce (Full Frontal Assault) now reach Intros. Further improvements are in the works to help few of these titles progress even further!

Resistance 3

With the above mentioned fix, this AAA Playstation 3 exclusive renders the opening cutscene but crashes before going ingame. Upcoming improvements are expected to benefit this title further!

Wheels of Destruction

This PS3 exclusive is now fully playable!

Sakigake!! Otokojuku-Nihon yo, Kore ga Otoko de aru!

Yet another weeb PS3 exclusive title progressed to playable this month!

Wakeboarding HD

Thanks to elad335’s MFC list fixes, this casual sports PS3 exclusive went ingame. This title however is prone to crashes and lose of audio. Do remember to disable the native UI during installation of game data to avoid crashes.

FIFA Street

Fifa Street, which is exclusively on consoles made it’s way ingame this month! Though FPS is at a stable 60, Fifa Street, like many EA Sports games suffers from broken physics which means you can’t move around properly. There is also some graphical issues where the players are missing parts of their bodies. So, despite a stable 60fps this game still needs more improvements before it can be considered playable.

Commits

Nekotekina

4705 – Added an option to disable SPU cache generation; Minor SPU ASMJIT and analyser improvements targeting safety (compatibility) and reducing the amount of garbage being compiled: Add ‘Accurate GETLLAR’ option in Debug tab and improvements to PUTLLUC/GETLLAR transactions to fix a regression on TSX-enabled processors; Added a progress dialog for SPU module compilation;

4742 – Fix returnable STOP, STOPD, SYNC, DSYNC in Safe mode and returnable STOP in Mega mode for SPU ASMJIT; Add ‘Accurate PUTLLUC’ option in Debug tab and minor changes in GETLLAR/PUTLLUC transactions; Fix cellGameContentPermit bug caused by 4721;

4750 – Implemented concept of big blocks for ASMJIT recompiler, see coverage in major improvements; Fixed mixing progress by building SPU cache after PPU; SPU ASMJIT now uses the same spu.log location as SPU LLVM; Improve spu.log disasm and trampolines which is now unified with SPU ASMJIT; Improvements to SPU LLVM with support for Mega/Giga modes, implementation of function chunks table, use of PHI nodes for value visibility across basic blocks and a simpler memory manager. Also, simplified code by using root config location for /dev_flash/ and hiding Emu.GetEmuDir() function due to the risk of misuse;

4801 – Updated LLVM commit to newer release, fixed SPU ASMJIT for SSE2 CPUs and restored /dev_flash/ config;

4804 – Fixes for booting PS1 Classics, see coverage in major improvements.

kd-11

4698 – “Simple” fix, includes FIFO reordering, depth scaling and MSAA fixes, see coverage in major improvements;

4752 – For the Vulkan renderer, Implemented type-agnostic transfer (e.g rgba16f to rgba8 memory) which fixes lighting in some situations, basic implementation of compute transactions to offload some calculations from the SPU and fixups for the RADV driver. Also, minor theoretical fixup for texture cache and improved read prediction for synchronized resources (WCB or blit engine) which significantly reduces the number of hard faults and lowers latency by eliminating unexpected readback;

4784 – Fix memory region pitch inaccuracy as region pitch of 64 (disabled) can also be used to indicate packed contents instead of the actual pitch which improves lighting in some titles. Also, fixes interaction of AA factors with lockable_region size;

4794 – Improves buffer synchronization logic which fixes ‘zoom’ when using WCB/CPU blit and upscaling resolution; Replace all memory block operations including endianness swapping and format conversions using compute for vulkan (the driver automatically does this step for OpenGL); Reimplement framebuffer access, loosen up memory relocking logic and implement forced reads if self-referencing active render targets; Update glslang to a much newer version which fixes broken SPIR-V.

elad335

4769 – Fixed the size of the MFC list elements read by ignoring the reserved bit. This PR allowed Hitman Sniper Challenge and Wakeboarding HD to move from Intro to Ingame!

4791 – Cleaned up some files, enabled shaders directory creation to be handled once on emulator initialization instead upon every shader creation;

4795 – Fixed the RSX base address which allows vsh.self to interact with the RSX and resolve related crashes;

4810 – Fixed regression to the RSX capture tool caused by the previous PR;

4825 – Removed second IO address translation and validation which makes RSX command reading more efficient.

isJuhn

4715 – Fixed cellGameCreateGameData temporary directory path where some games crash when using async cellfs to incorrectly attempt to access the temporary directory on dev_hdd0 instead of dev_hdd1. Setting the path for the temporary directory to dev_hdd0 allowed Aliens: Colonial Marines to go ingame!

4721 – Fixes a bug where the emulator would return the path to a non-existing gamedata folder when cellGameContentPermit was called on gamedata without calling cellGameCreateGameData first. Now in such cases, it will simply remove the content_permission. This PR allowed Game of Thrones: A Telltale Games Series to become playable, PES 2010 to go ingame and PES 2011 to reach Intro!

4806 – Certain titles by Insomniac games use unsupported methods for reservations. Since these titles always write with the same value (0xAAAAAAAA) when they fire an LR event, a workaround for this issue has been implemented. This PR allows Ratchet & Clank – A Crack in Time, Into The Nexus, All 4 One and QForce along with Resistance 3 and Fuse to move from Loadable/Nothing to Intro!

clienthax

4802 – Support for PS1 Classics, see coverage in major improvements.

Megamouse

4681 – Fixes some complaints for the Discord-RPC integration after its initial addition. It can now be disabled in the GUI settings and you can also customize the status for your idle state (when you don’t run any game but have RPCS3 open). You still have to restart RPCS3 to disable Discord-RPC completely though due to an upstream bug;

4709 – This fixes a bug where the Qt Dialog progress bars never reaches 100%, they always stopped at the last update before completion and stayed in that state until the game started showing its own frames. This was caused by a multitude of simple bugs; Also, switches the progressValue int pointer to int, handles file access error in cellSaveData by returning a verbose error instead of just fs::file is null, removes old (now unused) code for the wonderful PPU module compilation wall of dialogs, adds taskbar progress to all progressbar dialogs and introduced taskbar progress for the native UI in the thumbnail of the gs_frame (game window) which was implemented by adding a callback for the gsframe;

4739 – Adds counters to the “Find Dialog”. You can now see how many unique hits as well as hits in lines any of your searches in the log produce (use ctrl+f);

4729 – Fixes a few old bugs as well as new bugs that slipped in during 4709. Qt progress bars actually don’t update if you add a value that would make it exceed 100% (duh). So the new value was clamped to be inside the progress bar’s range. It also fixes a missed broken end condition in the native UI’s progress bars;

4740 – Added an option to create notes to a game by right-clicking it and choosing “Edit Tooltip Notes” which will open a dialog in which you can edit that game’s tooltip notes. Whenever you hover your mouse over a game’s title or name you will see the note displayed as well as the game’s name and title; Added an option to quickly copy the game name, serial or both in Name [Serial] string that can be used to quickly search for the game online or opening a new thread on our forums; Also, fixed some minor bugs with the game list;

4766 – Implements the “Enable TSX” combobox originally introduced by @NeoRetro in 4486. This combobox in the settings dialog let’s you disable or enable it at your leisure in order to benchmark the differences between the code branches, work around specific bugs or test for regressions. Also, some Haswell or Broadwell CPU models support TSX while some don’t, and others can only support it after unlocking it with a BIOS/microcode downgrade. If you have such CPU models with a microcode which has enabled TSX, you can set the TSX combobox to “Forced”, which will produce a warning that will let you decide if you REALLY want to enable it, since it may be harmful for your CPU. The state “Enabled” won’t work in such cases;

4777 – Simply adds the option to boot games by pressing enter or return while having an item selected in the game window which should help keyboard users;

4770 – Discord integration fixes and follow up for 4681. Fixed the new discord settings’ save behaviour inside the settings dialog which was applied regardless of accepting or cancelling the settings. Also, Discord code was moved to its own interface to streamline fixes to one place. Fixed incorrect behaviour in which the status often wouldn’t really update after you booted a game due to placing the update method in the wrong part of the boot process, which was never executed in some instances;

4783 – Fixup for 4709. Since the gs_frame’s taskbar progress will stay unchanged during a game session due to its members’ persistence, unlike the msgdialog, a reset function was added which is used whenever the native UI progress dialog closes;

4760 – Fixes the compatibility database download; Fix icon size actions not always being updated in the menu; Add more options to change trophy manager icon sizes (mousewheel, ctrl +, ctrl -); Disabling discord-RPC now uses Discord_Shutdown after upstream got patched; Adds “Delete SPU Cache” functionality; Use Qt to find and delete files; addresses issue 4725 (settings dialog tabsize); addresses issue 4717 (close config files).

hcorion

4685 – Fixes #4682, Fixes a bug where a failed branch appveyor build may take up the master versioning and future builds of that commit will fail to be renamed as there can’t be duplicated versions;

4669 – Unify Linux detection in CMake and Convert all spaces to tabs in CMakeLists.txt;

4699 – As AppVeyor has decided to no longer host binaries older than 6 months, windows binaries will be deployed through Github releases;

4744 – Fixed a null-deference which occured in the configuration dialog for evdev controllers;

4605 – Prefer GLVND for OpenGL rather than legacy and enable wayland for AppImages;

4780 – Make register editor dialog work in debugger.

scribam

4652 – cmake: simplify glslang integration;

4497 – 3rdparty: update stb_image.h and stb_truetype.h;

4741 – qt: rewrite syntax highlighter;

4745 – qt: fix asm instructions highlighting;

4720 – yaml-cpp: update submodule and build integration;

4748 – asmjit: update build integration;

4733 – libpng: use official repository instead of the fork from RPCS3;

4757 – zlib.vcxproj: remove PreprocessorDefinitions “Z_SOLO”;

4719 – pugixml: update submodule and build integration;

4778 – Allows RPCS3 to build on Linux when system zlib is not found and on Windows using CMake.

jarveson

4704 – Removed unnecessary discord debug libraries;

4728 – Fixes for 32 bit SPU elf that has compression flags, relocations that uses 0xff as an index value and trophies where the structs are zeroed out before use. Also fixes image_in and swizzled CPU transfers. This PR allowed Yakuza Kenzan and Counter Strike: Global Offensive to finally go ingame!

VelocityRa

4727 – Refactored the Performance Overlay and also added options to customise the position, font, margin width and background opacity of the overlay.

Farseer2

4764 – Fixed a bug where sys_spu_thread_connect_event would fail valid queues. This PR allowed WRC 5 to go ingame and Dynasty Warriors Gundam to become playable!

JohnHolmesII

4697 – Update Arch install directions in the README.md and typo fixes;

4800 – Update the links to the precompiled LLVM and additional libraries in the README.md.

Cabalist

4828 – Fixed python white spaces and switch to .format() strings.

jbeich

4788 – Unbreak build on FreeBSD.

Zangetsu38

4762 – Remove unnecessary .gitignore files and win32 config from the repo.

Talkashie

4751 – Disable GPU Texture Scaling by default.

jjsat

4732 – Fix VSLO and VSRO in the LLVM PPU recompiler which helps improve vertices rendering in various titles including Deus Ex Human Revolution and Batman Arkham Asylum.

al3xtjames

4707 – Add additional dependencies needed for evdev and ALSA support for Fedora to README.md.

Dravonic

4677 – Parallel shader cache loading using all available threads during the initial shader compilation process for the Vulkan renderer.

Daniel-Valentine

4658 – Delay GDB server launch until emulator run to fix few segfaults.

ikki84

4634 – Return offline instead of cell_ok for sceNpManagerRequestTicket2 when the PSN connection status is offline.

Closing Words

If you like in-depth technical reports, early access to information, or you simply want to contribute, consider becoming a patron! All donations are greatly appreciated. RPCS3 has two full-time coders that greatly benefit from the continued support of over 800 generous patrons.

Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by Asinine, eladash, HerrHulaHoop, Megamouse, nitrohigito and Talkashie.

Progress Report: May 2018

RPCS3 continues to see major improvements over the month of May, with Nekotekina implementing SPU LLVM (still WiP) and kd-11 continuing with improvements to RPCS3’s RSX emulation. More major AAA exclusives have also started to go ingame for the first time! We also saw new contributors join in and make much appreciated improvements to RPCS3.

In addition to the following report, further details of Nekotekina and kd-11’s work during May and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2018-05-05)
Status update from Nekotekina (2018-05-14)
Status update from kd-11 (2018-05-20)
Status update from Nekotekina (2018-05-29)

Table of Contents

Major Improvements
Games
Commits
Conclusion

The Nothing category is now at an all-time low, reducing by just over half, work is already being done to make this category shrink even further! The Playable category has had another nice increase, as more Playable titles were found. Furthermore, many entries for the same Game Media on the list were merged (due to new reports for different regions being submitted these past months), so the overall game count has decreased, even though there were new unique submissions. For a more detailed look, you can view the compatibility history page to see exactly which games had their status changed this month.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (May 2018)

On Git statistics, 23 166 lines of code were added and 6 732 were removed by 17 authors.

Major RPCS3 Improvements

SPU LLVM Recompiler (#4504)

While still experimental, Nekotekina implemented SPU LLVM Recompiler during May! It may not currently work with many games, but the ones that do work saw a good improvement with performance and audio. For example, Wangan Midnight which works with SPU LLVM has seen audio stutter disappear, with other titles also receiving note-worthy improvements from SPU LLVM such as Drakengard 3, Killzone 3 – which now goes ingame for the first time – and Tales of Vesperia. Be sure to test your games with it and report back if you find anything interesting!

Shader Analyser (#4505)

On the first of May kd-11 begun implementing a Work In Progress shader analyser which will help reduce the amount of shaders RPCS3 compiles. This is done by gathering metadata during the compilation of a shader and using that information to check shaders against bound textures. By doing this, unbound textures will simply be skipped instead of having to be decompiled just to check if they are actually being used.

kd-11 also made RPCS3 assume that if there is no L2 cache eviction then the fragment program micro code has not been updated and to do the same for some registers that control vertex programs. As the hardware would not see the changes anyway, there is no need for RPCS3 to do extra work. This improves performance in titles such as Far Cry 2, Prince of Persia and Metal Gear Solid 1 by a significant margin.

This lays the foundation for more future improvements that should eventually eliminate shader compilation stutter entirely. This Pull Request also fixed/improved shadows in Assassin’s Creed, Saw 2 and Prince of Persia.

Texture cache Improvements (#4611)

By re-implementing RPCS3’s texture cache to take advantage of the memory mirror support added by Nekotekina, kd-11 fixed the following issues; Flickering cutscenes in Uncharted, Sly Cooper Thieves in Time, Tekken Tag Tournament 2 and broken particle and alpha effects in Beyond Good & Evil HD. This change also improved RPCS3’s stability.

Memory protection fixes and RSX Improvements (#4649, #4671)

kd-11 made some fixes to RPCS3’s memory protection which fixes flickering and missing objects in some games especially if Write Color Buffers was enabled. This also reduces the need for Strict Rendering Mode in various Naughty Dog games such as The Last of Us and Uncharted games. This means that you can now use resolution scaling in The Last of Us! Click on the image below to view it at the full resolution.


You may have noticed already that the visuals are not very impressive, even though it’s running at 4k. This is caused by issues with post-processing. kd-11 plans to work on improving this in the future.

kd-11 also fixed resolution scaling in God of War 3 by constantly pinging the RSX thread while in the spinwait state to check for cyclic dependencies. This allowed the game to no-longer run in software mode which resulted in scaling being fixed! kd-11 also fixed a regression where God Of War 3 required the debug option “Force CPU Blit” to be enabled which lowered performance and caused confusion among users.

These improvements among others can be seen in the video below.

RSX Capture/Replay (#4510)

On the RSX front Jarveson implemented the beginnings of a fairly useful debugging feature, capturing and replaying! For those that have ever tried to debug graphics issues, you know that RenderDoc is invaluable when trying to track down and fix them. The shortcomings with it, however, is that it doesn’t tell the whole story from the emulator side, it leaves out what the game is actually telling the RSX to render. This feature aims to fix that. By capturing the RSX commands and memory needed for a displayed frame, we’re now able to replay and display that exact frame in the emulator. A developer can now see the full path from game to RSX to screen, without having to waste time on trying to reproduce the error. This can also help with regression testing as well, as replays can be compared between builds. It should be noted there are a few restrictions and issues with it in its initial creation, check out the pull request description for more info.

Vulkan Memory Allocator & Timing Fixes (#4579, #4635)

Pauls-gh discovered that Tales of Vesperia was bottlenecked by the GPU in certain areas and went on to profile the code. He then found that Vulkan video memory allocations were very slow. This led to adding the open source Vulkan Memory Allocator library, with this library Tales of Vesperia sees a performance improvement of up to 100%.

Shortly after, Pauls-gh looked into a long standing issue with Tales of Vesperia where character animations would play in slow motion. After finding that Linux did not have this issue, he used Microsoft Concurrency Visualizer and found that the game was spending a lot of time waiting on conditional variables (cond_variable::imp_wait).

This led to discovering that the OS wait (NtWaitForKeyedEvent) was not microsecond accurate. Windows can only sleep using a resolution of 0.5 msec. After modifying the PS3 function sys_timer_usleep to skip sleeping if the time was less than 300 usec character animations were finally running at the correct speed on Windows.

However, this wasn’t hardware accurate as kd-11 quickly discovered after running hardware tests on a real PS3. To address this, kd-11 modified the RPCS3 implementation of sys_timer_usleep to make it usec accurate with real PS3 hardware then tuned values to ensure performance was not lost on low-end CPUs (committed as part of #4661).

This lead to massive Windows performance improvements in Nier, Tales of Vesperia, Atelier Escha & Logy Alchemists of the Dusk Sky and more! Nier in particular went from single digit frame rates to a locked 30 FPS on many machines which resulted in this exclusive becoming playable!

Here are a few comparative screenshots of Tales of Vesperia where the image on the left has Vulkan Memory Allocator disabled and the image on the right has Vulkan Memory Allocator enabled:

Performance Overlay (#4620)

This month VelocityRa added a much requested feature, a Performance Overlay. Work on the overlay began after nitrohigito opened a feature request (#4500) in April. After 2 months of fleshing out the scope and design of the overlay, the PR was finalised and merged this month. The Overlay presents various information such as:
1. Total CPU Usage along with the breakup between the PPU, SPU and RSX emulation
2. Thread count for PPU, SPU and RSX emulation
3. FPS and frame time
4. RSX Load

The Overlay has a few configurable options, the most important of which is the Detail Level.
It has four such presets, namely ‘Minimal’, ‘Low’, ‘Medium’ and ‘High’, each with increasing levels of detail:

Since the Performance Overlay made use of the native UI, kd-11 refactored and fixed various issues present in it to make it easier for other developers to cleanly manage adding new dialogs and interfaces without worrying about object management (#4623).
Also, Megamouse handled the Qt aspects by adding a GUI tab to the settings dialogue and exposed some Performance Overlay options to it (#4673).

Revamped Trophy Manager (#4604)

Thanks to the work of flash-fire, RPCS3 gained a trophy manager back in October 2017. While the initial implementation focused primarily on the design and functionality, the trophy manager did come with a collapsible column structure that maintained both the game and trophies in a single table. While this was a welcomed addition, the trophy list proved arduous to use with multiple games in the list. To fix this, Megamouse completely rewrote the list structure of the trophy manager this month! The Trophy Manager now uses two tables, one for the games and one for selected game’s trophies. You can switch to a game simply by selecting it with either mouse or keyboard and thus display its trophies to the right.

Also, game icons have now been added to the games table! Just like the trophy icons, these icons are also resizable. You can also use the handle between the tables to resize the tables and even hide either of them. A dropdown was added to the leftmost toolbar in order to make it possible to still choose games while the games list is hidden.

Games

NieR

As mentioned earlier in the report, NieR saw a massive performance boost on Windows specifically due to some timing fixes. Instead of single digit framerates on Windows, you can now expect a locked 30 FPS even on a mid-range CPU. This made NieR playable for the first time! If you want to try running the game for yourself, just remember to enable Write Color Buffers (GPU Settings Tab).

Lollipop Chainsaw Massacre

While this title has been playable for around a month now, graphics have recently improved and it no longer requires strict rendering mode meaning you can play at high resolutions without breaking visuals!

Shadows of the Damned

Another console exclusive game is now Playable, this one runs quite well just remember to keep the aspect ratio set to 16:9 and Anisotropic Filtering on Automatic if you want to try it out for yourself!

Killzone 2 & 3

With the recent improvements to SPU ASMJIT Recompiler by Nekotekina, these 2 AAA PlayStation 3 exclusives finally go ingame!

Silent Hill Downpour

An issue with cutscenes that also affected Ni No Kuni which was fixed in April also allowed Silent Hill Downpour to get past the prologue without a work-around. This change, among other graphical and performance improvements have allowed Silent Hill Downpour to be considered playable!

NCAA Football

NCAA Football 10 and 11 and most likely many more titles go ingame after user “sftt” fixed an underflow issue in sys_memory.

Pro Evolution Soccer

Thanks to recent improvements to elad335 to the memory mapping alignment, various PES games now reach Intro.

Disorder 6

This console exclusive was previously ingame due to graphics and audio glitches. Recent testing has shown that this title is playable!

Ketsui Kizuna Jigokutachi Extra

Another arcade console exclusive title which made its way to playable.

Muv-Luv Photonflowers and Photonmelodies

These exclusive VNs are now playable. More weeb games!

Commits

As always, this is not a complete list of PRs or commits, nor does it necessarily list every single thing done by a given PR. For a full list, see PRs merged in May. Many of the unmentioned PRs are simply small updates, fixes, and other small improvements, sometimes only to the GUI.

Nekotekina

4504 – SPU improvements, see coverage in major improvements.

4553 – Improves Travis automatic build systems, SPU analyzer and code generation.

4565 – Fixes some regressions caused by the previous PR.

4580 – Asmfixes, rewrite of the TSX transactions’ logic in assembler which improves the speed and consistency between different platforms and compilers. Also fixed a bug in Linux where some games would only show a black screen #4460.

4603 – Asmfixups, fixes some regressions caused by the previous PR.

4622 – Diagnostic, fixes some more regressions caused by #4580;

4631 – Perf fixes, Improves emulator for non-TSX processors. This work is intertwined with eliminating deadlocks in RSX.

4646 – Perf fixups, Continuation of the previous PR.

4664 – Insignificant fixes, rewrite of SSSE3 path to use ASMJIT to generate SSSE3+ code at runtime. Helps AppImages which were previously compiled without SSSE3 support. Also, LLVM compilation progress dialog will now use a single dialog for all threads!

kd-11

4505 – rsx: Stuff, Introduces the base for the gpu programs analysis and management; Properly works around two AMD driver bugs, one Vulkan bug relating to primitive restart and another OpenGL bug caused by broken generation of gl_vertexID in some situations; Properly initialize CC registers to fix some poorly written assembly shader ucode; Implement flow control. This implements properly jumps and function calls (CAL, BRI, RET, etc). This adds proper support for subroutines among other improvements.

4623 – Refactor of the Native UI overlays system, see coverage in major improvements. This PR also fixes some annoying flickering with native interfaces or in some cases invisible interfaces.

4628 – Fixup: Overlays, addendum to the above PR.

4611 – rsx: Fixes, Reimplement texture cache to take advantage of the memory mirror support and minor fixes to logging to reduce log spam.

4649 – rsx: Fixup, Memory protection fixup which fixes flickering and missing stuff in some games especially if Write color buffer is enabled; Optimize a check in main RSX loop to restore performance lost in the above PR; Fix OpenGL Write color buffer regression (‘unreachable’ assert).

4661 – Stuff, Disable thread scheduler for intel CPUs; Fix a race condition causing crashing in overlays; Promote FIFO optimizations to stable which prevents the feature from getting disabled by when using Strict rendering mode and allows Strict rendering mode to work with usable performance. A debug option has also been added for developers; Finish reimplementation of sys_timer_usleep.

4671 – More stuff, Evade semaphore acquire deadlock by constantly pinging RSX thread during spinwait to check for cyclic dependencies; Simplify task queue management and use yield instead of busy_wait to reduce audio stutter on Intel i5 processors; Fix silly bug causing native UI crashes when trophy popup closes.

4679 – Fixes a leak in performance overlay causing ever-increasing compiled buffer size with ridiculous redraw and rework of Vulkan overlays handling with better memory management.

Megamouse

4550 – Moved the additional Qt plugins (like the folders: bearer, imageformats, styles and platforms) into a subfolder “qt/plugins” using windeployqt and a new .conf file. The old folder structure can still be used to preserve backwards compatibility;

4562 – Fixes a bug in the previous PR which resulted in broken Appimages. Paths for windeployqt are now moved to an extra windows.conf file;

4594 – The prior gitignore file did not always exclude all files inside the bin directory. This resulted in config.yml files, logs, 7z and many more to show up in the unstaged files in gitkraken. A rather simple rewording fixed this issue;

4407 – Includes notable new feature additions to the gamelist along with other minor fixes. First of all, a new column has been added to the gamelist: “Move Support”. By checking a flag in the param.sfo this column shows us if a game supports the move controllers, which will hopefully soon be added to the roster of usable peripherals. Currently this is very useful in order to sort for games that can or can’t be played at the moment. Another bigger change (apart from some rare GUI crash fixes and optimizations) is the complete overhaul of the gamelist data model. This fixed a lot of minor issues, like weird scrolling after sorting or game additions/removals, wrongly selected items after similar interactions and last but not least wrong sorting results;

4604 – Major rewrite of the Trophy Manager, see coverage in major improvements. Further improvements were also made to the gamelist such as fixing size issues regarding the gamelist headers, rework the way their settings are saved in order to fix future inconsistencies and more;

4378 – You can now just press any “Play” option on starting RPCS3 in order to boot your last played game. This is cool if you play or test the same game repeatedly, whether if it crashes the emulator, whether you use different builds, or maybe just want to backup logs. You could do this before already with a shortcut: ctrl+1. Also, the log message severity for “Recent Game Not Valid” has been moved from Error to Warning (only the common occurrence that happens when you delete or move games);

4617 – Bugfixing for 4594, 4604 and 4407;

4621 – Bugfixing for 4604;

4397 – Fixes an error that prevented “Battle Princess of Arcadias” to proceed ingame. A savedata function falsely aborted when an allowed (albeit technically incorrect) status was returned by a callback. Allowing the function to progress and only filtering out actual errors was the correct solution to this problem;

4637 – Fixes various minor things in the gamelist and also changes the size ratios of a few main window elements. For example: The log frame is now smaller when you first boot up RPCS3 and also the gamelist columns will have a slight offset to each other;

4673 – All UI settings in the Emulator tab have now been moved to a dedicated GUI tab. This was done primarily to accommodate Performance overlay options of VelocityRa in the Emulator tab. It now has a checkbox that enables the Performance Overlay and its other options: a combobox for the 4 detail levels from Minimal to High, a slider to change its update interval in milliseconds and a slider that controls the font size. Hopefully this part of the settings dialog will come in handy!

elad335

4660 – Rsx: Naughty fixes, Fixes RSX IO addressing assignment when one of the IO address ranges has been deallocated which allows the IO mapper to return the correct IO offset values.
This fix allowed exclusives such as Uncharted 2 and Uncharted 3 to finally go ingame and also allowed The Last of Us to progress a little further!

4672 – Fixes memory mapping alignment check by checking the address alignment via the alignment of its page. This fix allowed five Pro Evolution Soccer games to go from Nothing/Loadable into Intro!

jarveson

4510 – rsx: initial capture/replay functionality, see coverage in major improvements.

jbeich

4532 – Unbreak build on FreeBSD;

4540 – Make sure assertions (e.g., in LLVM) are disabled by default;

4596 – Unbreak build on FreeBSD;

4655 – Unbreak build on FreeBSD.

Nicba1010

4650 – Vulkan SDK Mirror, Setup mirror for Vulkan SDK executable which were previously causing automatic builds to fails due to a download cap set.

scribam

4514 – hle: make cellSubDisplayInit returns CELL_SUBDISPLAY_ERROR_ZERO_REGISTERED, Simulates that a remote play device is not connected. Improves Class of Heroes 2G and Guacamelee! (Playable Demo);

4572 – Fixes some typos in RPCS3’s code.

4574 – Improved cellSync2 error checks and logging (HLE).

4567 – Housekeeping, Removes some redundant code relating to rpcs3-tests and ps3emu_api;

4593 – Compilation fix for GCC 8;

4578 – Cmake improvements;

4577 – Travis improvements, Linux builds now rely on Vulkan packages which negates the need to checkout the submodule. Also, leftover instructions related to the usage of the QT installer has been removed;

4608 – 3rdparty: update Vulkan integration, Remove the Vulkan-LoaderAndValidationLayers submodule/project to directly use the official Vulkan SDK;

4639 – Link dynamically with the Vulkan loader, Avoids linking to the static library which may cause issues. With this PR, Vulkan is now compatible monitoring tools such as RivaTurner/ MSI afterburner.

pauls-gh

4579 – Timer resolution fix for Windows 10 update 1803 performance regression, see coverage in major improvements.

4635 – Vulkan memory allocator performance enhancement, see coverage in major improvements.

VelocityRa

4618 – Fix Windows build;

4620 – Performance Overlay, see coverage in major improvements.

hcorion

4581 – Make Appimages work and fixes xxHash issues.

4588 – Re-enable LLVM for travis and build AppImages with LLVM 6.

creeperjedi

4564 – Added “(experimental)” next to the SPU LLVM Recompiler.

tlm-2501

4558 – Add description for SPU LLVM.

isJuhn

4539 – Fix setParam in cellHddGameCheck which allows Record of Agarest War to go ingame.

AniLeo

4526 – gui/themes: Refactor Kuroi (Dark).

Orphis

4507 – CMake build optimization.

Maxetto

4490 – Update some lv2 syscall names.

sftt

4399 – Avoid illegal available_user_memory (<0 or underflow) in sys_memory.cpp, which allowed Shining Resonance, Everybody’s Golf: World Tour, and NCAA Football 10 and 11 to go ingame!

Closing Words

If you like in-depth technical reports, early access to information, or you simply want to contribute, consider becoming a patron! All donations are greatly appreciated. As of last month, RPCS3 now has two full-time coders that greatly benefit from the continued support of over 700 generous patrons.

Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by Asinine, HerrHulaHoop, Jarves, VelocityRa and Megamouse.

Progress Report: April 2018

April has been a month full of advancements in nearly every aspect of RPCS3. Performance boosts, graphics fixes, and compatibility improvements are plentiful.

Table of Contents

Major Improvements
Games
Commits
Conclusion

On Compatibility statistics, we can see a huge increase in Playable titles. A lot of games were retested, and many Ingame or below ones were found to now be Playable. There was also a considerable decrease on the amount of Loadable games and a slight increase in games classified as Nothing, as new games were added into the category.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (April 2018)

On Git statistics, 7 018 lines of code were added and 4 535 were removed by 12 authors.

Continue reading Progress Report: April 2018

Progress Report: March 2018

March has been an amazing month full of progress for RPCS3. This month marks the first time that games in the “Nothing” category fell beneath 1%! We had massive strides in graphical accuracy, along with improvements in SPU performance and game compatiblity. From giant pull requests that generated tons of hype, to a small typo fix that had unexpectedly wide-ranging impacts, this month has had it all.

Table of Contents

Major Improvements
Games
Commits

Below are the usual compatibility database statistics, with all metrics moving in the right direction. The tiny number of games left in the Nothing category is quite exciting, especially considering this was the largest category not long ago.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (March 2018)

Continue reading Progress Report: March 2018

Progress Report: February 2018

RPCS3 has reached version 0.0.5! The full change log since 0.0.4 can be read here, but the highlights are massively improved compatibility (nothing section is down to 1.34%!), better AVX-512 and AVX2, as well as input improvements and profiles.
We also saw the end of PSP2 emulation in RPCS3. While it remains alive in a separate branch, the Vita emulator has been removed from master in light of the Vita3K project.
kd-11 also implemented a native user interface, allowing for games to be played fully with a controller, with the small exception of the on-screen keyboard, which doesn’t exist yet.

This report will focus on the changes that happened in February, which include some exciting work slimming down the nothing and loadable sections. RPCS3 is within reach of having every tested PS3 game be at least in the loadable section, if not the intro section!

Table of Contents

Major Improvements
Games
Commits
Upcoming
Conclusion

On compatibility database statistics, there’s not a big difference from last month as less games were tested, even though there were massive changes to the emulator (expect a beefier March report!). We can see an overall ~0.20% decrease on all categories except Ingame, which increases by ~0.80%. Note that we currently only have 34 games left on Nothing, and plan on making that number even smaller very soon!

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (February 2018)

On GitHub statistics, 6 939 lines of code were added and 36 937 were removed through 43 Pull Requests by a total of 15 authors.

Continue reading Progress Report: February 2018

Progress Report: January 2018

Welcome back to the first progress report of the year. Quite a large number of fixes and improvements have been made already, and as you can see below from the big compatibility bump we are starting the year off strong. kd-11 has made some major improvements that increase stability and functionality including increasing performance with Ryzen and implementing a Native UI to RPCS3. While Nekotekina has continued to review people’s changes, making sure everything was in a good enough state to be merged, he continued to work on the core of RPCS3. Megamouse has also made many improvements to the way RPCS3 handles input allowing a bunch of games to progress further and he also fixed other issues with the UI.

Table of Contents

Major Improvements
Games
Commits
Upcoming

On the compatibility database statistics, over a hundred games moved out of the Nothing, Loadable, and Intro sections and in to Ingame or Playable. In addition, the Nothing section has now fallen below 2% of tested games, and hopefully Loadable will follow next. This means RPCS3 is on the right track to ensure accuracy and compatibility with the largest number of titles possible.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (January 2018)

On GitHub statistics, 7,454 lines of code were added and 2,160 were removed by a total of 14 authors.

Continue reading Progress Report: January 2018

RPCS3 Begins to Emulate Several Awaited AAA Exclusives. Here’s How We’ve Done It!

Yes, you’ve read correctly. Many of the much awaited exclusives are now finally starting to be emulated by RPCS3. In this blog post, you’ll learn which games we know to have improved and how we’ve done it.
But first, check out this awesome teaser:

Table of Contents

SPU Improvements by Jarves
RSX Improvements by kd-11
List of known Improved Games
Closing Words

SPU Improvements by Jarves

There were two main changes to the SPU emulation that brought us to this point of allowing so many newer titles to progress past ‘Intro.’ Let’s take a quick look at both individually.

SPU Interrupt Fix

Many titles in RPCS3 ‘hang’, but in the case of the titles mentioned above, they do not actually crash, and the fps counter would still change with just a black screen. This normally would be mistaken for RPCS3 just being slow and the game taking a bit to load, but opening up the debugger in RPCS3 tells a different story. The games would loop over the same code on both the PPU side and SPU side. In this case, they are waiting on something, but what?

‘The Last of Us’. More like – ‘The Last Loop That Will Ever Execute’
Moving the PC to address 0 of an SPU thread explains more. Most games will have just 0 (null) written there, but some actually have a branch there.

Continue reading RPCS3 Begins to Emulate Several Awaited AAA Exclusives. Here’s How We’ve Done It!

Progress Report: October 2017

October was a huge month for RPCS3. Our lead graphics developer, kd-11, implemented High Resolution Rendering and made many improvements to the emulator. While Nekotekina made some general improvements to the emulator, which will be discussed in this report, he also bought some parts and assembled a new computer with a future-proof CPU which will allow him to debug RPCS3 more efficiently, faster, and allow him to make sure newer instruction sets like AVX-512 work properly with RPCS3 when the right time comes. You can find more details on the latter and also discuss further on it at https://www.patreon.com/posts/pursuing-avx-512-15151688

Table of Contents

Major Improvements
Games
More Improvements (Commits)
Upcoming
Conclusion

Starting off with the compatibility statistics as usual, the Intro/Loadable/Nothing categories keep growing small and Ingame/Playable titles keep increasing. We’ve also hit 500 playable games this month!

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (October 2017)

Looking at the GitHub statistics, 16 authors have pushed 131 commits to the master branch. Here 180 files have changed and there have been 12,626 additions and 5,068 deletions of lines of code. Below some of the major improvements from these code changes are summarized.

Continue reading Progress Report: October 2017

Lead Graphics Developer kd-11 Is Joining Nekotekina’s Patreon

Veteran RPCS3 graphics developer kd-11 is joining Nekotekina’s Patreon. kd-11 has worked on RPCS3 since January 2016 and has since then fixed countless bugs in all graphics renderers and implemented a lot of missing functionality. Without his dedicated work, RPCS3 would certainly not be where it is today.

As you know lead core developer Nekotekina, who has worked on RPCS3 since 2013, launched his highly successful Patreon in January this year. Thanks to the overwhelming amount of support, he has been able to work on RPCS3 full time for 5 months now and the progress he has done is staggering. The RPCS3 team has discussed the best course of action for the road forward and have decided that the time is now right for kd-11 to join Nekotekina’s Patreon in order for him to obtain hardware necessary for development and testing.

The Purpose

As kd-11 works on the graphics side of things, especially the renderers using OpenGL, Vulkan, and D3D12 (Direct3D 12), modern hardware is required for development and testing. Ideally, he needs a modern AMD, Nvidia, and Intel GPU to easier identify bugs specific to one platform. Furthermore, kd-11 works as a consultant and therefore has the possibility to accept or turn down certain jobs. If support on Patreon reaches a certain level, he may be able to spend more time on the project.

As of now the goal is obtaining necessary hardware for development and testing:

  • – A modern AMD GPU that supports DirectX12 and Vulkan.
  • – A modern Nvidia GPU that supports DirectX12 and Vulkan.
  • – A modern Intel CPU where the iGPU supports DirectX12 and Vulkan

Roadmap

With the support of generous patrons kd-11 will be able to work on the following tasks:

Short term

  • – Investigate game specific bugs like the overly bright bloom in [redacted] or the overly dark lighting in Demon’s Souls.
  • – Improve performance and compatibility of all renderers.
  • – Enable Vulkan on Linux, which will bring a tremendous performance improvement for some games.
  • – Implement missing functionality that will fix various bugs in different games. For example, the broken shadows in many games including Demon’s Souls depend on this task.

Mid term

  • – Properly support some PlayStation 3 features like MSAA that are lacking at the moment.
  • – Implement RSX reports which are missing at the moment.

Long term

  • – Enable higher rendering resolution to play games at any resolution, for example 4K instead of 720p in Demon’s Souls or [redacted].
  • – Enable extra graphics options to support extra rendering features, such as custom anti-aliasing modes.

Continue reading Lead Graphics Developer kd-11 Is Joining Nekotekina’s Patreon