Progress Report: September 2017

September was quite the exciting month. Our lead graphics developer, kd-11, made major improvements to graphics emulation while Nekotekina, among many things, implemented initial networking support. This progress report will provide an overview of major changes and games that improved.

Table of Contents

Major Improvements
Games
Commits
Upcoming
Conclusion

First of all, here are the compatibility statistics for this month. As usual the category of games that do nothing is shrinking while more and more games become playable.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (September 2017)

Looking at the GitHub statistics, 15 authors have pushed 131 commits to the master branch. Here 216 files have changed and there have been 9,185 additions and 7,028 deletions of lines of code. Below some of the major improvements from these code changes are summarized.

Major RPCS3 Improvements

The big kd-11 PR (#3458)

This is the pull request of the month. So many things are improved it would be a daunting task to cover it all in detail. First of all, watch the video above that showcases various improvements in various games in just slightly over a minute. It should give a good idea on how major these improvements are. In short this PR fixes various bugs, improves code quality, enables GPU Texture Scaling on Vulkan, and otherwise improves Vulkan performance as well. In more detail this PR contains:

  • Fixes for bit blit in OpenGL. Bit blit is the process of combining two or more bitmap images (think ordinary 2D images) into one. The process could be as simple as putting one small image on top of another, or it could be a more advanced combination. Either way a lot of 2D fighting games among many other games make heavy use of this. As can be seen in the video above these have also improved drastically.

  • A bit blit engine was implemented in hardware with Vulkan. This is a major performance improvement.

  • It is now possible for the graphics thread to try and recover in the event of receiving unknown commands. This is actually possible on a real PS3 and works around many strange crashes in different games. These could be emulation bugs, but they are more often than not actual game bugs we can now work around the same way a real PS3 would.

  • Draw call batching is now implemented. There were a lot of very slow games that would throw thousands of small draw calls at the hardware. This significantly speeds some games up.

  • Implements a new method of Write Color Buffers that is always activated. This new method is much faster and is responsible for performance improvements in games that required Write Color Buffers to be enabled. The old method downloaded an image from the GPU, resized it and then reuploaded it which was not a very efficient use of PCIe bandwidth. While many games that required Write Color Buffers now work perfectly with the new method. Demon’s Souls and some other games still require the slow method, which is still available in the GPU settings. Users can check if the new method works for their games that previously required Write Color Buffers by simply disabling it.
    There are two reasons why games require Write Color Buffers, the first is when RSX is asked to scale a texture that exists in memory, and the second is when Cell wants to read values from the GPU. For instance, how bright the current frame is. The CPU has to read the pixels and calculate averages.
    The first situation is now permanently handled from within the GPU without ever reading the pixels to the CPU which is why it’s no longer needed. The second case is the one Demon’s Souls and UE3 games use. There is no workarounds for this and the data must be read back from the CPU.

  • Fixes some shader decompiler bugs and therefore crashes in, for example, some Dragon Ball Z games.

  • Compatibility with Mesa on Linux when OpenGL is used is now improved.

  • The groundwork for high resolution rendering is more or less completed now. The real work of implementing it can now start.

  • Refactoring and bug fixes in various areas all over graphics emulation and renderers.

Below we see improved graphics in GTA V and Red Dead Redemption. Specifically shadows are now working as they should in both games, and in GTA V characters also render correctly without any strange colors.

Lastly, here is The Legend of Heroes: Trails of Cold Steel running at full speed (30 FPS) with Vulkan on Linux. This game requires the hardware blit engine to perform more than a few frames per second and therefore only worked well with OpenGL earlier. But the Vulkan renderer is faster overall and with hardware blit and also draw call batching in place it now runs very well. Also take note that graphics including shadows render perfectly now.

Networking (#3503)

An image says more than a thousand words. As can be seen both above and below there is now some network support in RPCS3! But one thing must be stated clearly already: This does not mean that PSN is working. The vast majority of commercial games can not (yet?) play online. The point of implementing the network syscalls used by the libnet module in the PS3 operating system is to fix various issues such as games that connect to some server to download news about patches and DLC. This could crash before since it was simply not properly handled.

Implementing libnet was quite the undertaking that took Nekotekina several weeks of reverse engineering and coding. It even gave birth to the “ugh” meme on our Discord server. However, with the vast majority of work now completed, some users have already found improvements in the following games: Scott Pilgrim vs. The World, Portal 2, Resident Evil: Operation Raccoon City, Saints Row: The Third and Saints Row IV. These changes are expected to help even more games and applications in the future.

Improvements to automatic library loading (#3454)

This Pull Request by Nekotekina includes rewrites for most standard libc calls, which improves compatibility when using Automatically Load required libraries when loading firmware modules. Do note that if you use Load liblv2.sprx only, it will instead use the firmware’s version of these calls, which is preferred for increased accuracy.

It also includes some miscellaneous file system fixes and log fixes. For example instances of games failing to find files that actually exist got fixed with this PR. One such games is Unreal Tournament 3, but there are probably other instances as well.

Games

So many games were improved this month that it would be impossible to cover all of them. Instead, here are 11 select games or groups of games that are either interesting in some way, popular with our users, or that represent a certain change in a clear way.

Armored Core: Verdict Day

Armored Core: Verdict Day went from nothing at all to going ingame thanks to a combination of graphics improvements by kd-11 and library loading fixes by Nekotekina. It is slow, buggy, and graphics are mostly broken, but this is a great first step in getting the Armored Core games by From Software working.

Beyond: Two Souls

Beyond: Two Souls previously got ingame but rendered practically no graphics and quickly crashed. Thanks to the graphics emulation improvements by kd-11 it now renders most graphics mostly correctly, though there are still issues like broken depth of field present. However occasionally it now looks alright and even runs at double digit frame rates (and at other times it slows down 1 FPS).

Dante’s Inferno

Thanks to various commits by kd-11, Dante’s Inferno is now beatable in RPCS3. Along with massive performance improvements, this game saw a fix to the “exploding vertices” that plagued it previously. Even on a fourth generation i5, as seen in the video above, the game sticks very close to 60 FPS throughout. User Geovanny helped with recording the video above and also played through the entire game proving it to be beatable as seen below.

Dengeki Bunko: Fighting Climax

Thanks to the graphics emulation improvements by kd-11 this 2D fighting game, and really every other game in the same engine, now performs very well and also looks mostly correct. Previously these were slow and had severe graphical issues like completely missing sprites. They are now more or less playable, at least on fast hardware.

Journey

Thanks to miscellaneous SPU improvements and graphics emulation improvements by kd-11 Journey now goes ingame, previously hanging right before doing so. However it currently only works with the precise SPU interpreter which is the slowest thing on earth and therefore we are looking at something like one frame every three seconds even on a Ryzen 7. But the good news is that one can do some debugging with the precise SPU interpreter as reference to find the broken instruction(s) in the SPU recompiler. The SPU recompiler is significantly faster than the slow and precise interpreter, that 0.3 FPS could easily turn into something much more playable.

Ridge Racer 7

Thanks to another vague “fixes” build from Neko, Ridge Racer 7 is now ingame in RPCS3! While the graphics are fairly accurate for the most part, there are still some “exploding vertices” and texture flickering going on. Unfortunately and perhaps more seriously, the game rarely sees over 7 FPS for the time being.

Rune Factory: Tides of Destiny

Rune Factory saw major improvements this month! Previously, this game would die shortly after getting to the loading screen. Fortunately, one of Neko’s SPU improvement commits made this game almost entirely playable. Unfortunately, saving is still partially broken. One can load saves imported from a real PS3, but RPCS3 fails to create new save files.

Sly Cooper: Thieves in Time

Sly Cooper saw improvements to graphics and performance thanks to graphics emulation improvements by kd-11. Among many issues instances of things having the wrong color were fixed. The game would be playable on a fast CPU if not for completely missing audio. Below we also see the game running in an ultra wide screen aspect ratio thanks to a patch.

Sword Art Online: Lost Song

Sword Art Online: Lost Song saw major improvements to graphics thanks to work by kd-11. Various problems such as the game previously being squashed down into a small segment of the window was fixed. Thanks to these improvements the entire game was beaten by user Illynir in 30 hours with not a single crash or freeze occurring. Except for some incorrect colors in the first area of the game, it ran flawlessly at a locked 30 FPS on an i5-4670k, and looking at CPU usage even an i3 might work just as well.

Tales of Berseria

Tales of Berseria benefited from some of kd-11’s fixes this month. As you can see in the screenshots above, trees and their leaves now cast shadows as they were intended to. Before kd-11’s work, there were no shadows to speak of.

Tekken 6

Tekken 6 became truly playable this month thanks to the work done by kd-11. In the above video we see several matches with perfect graphics, including now fixed shadows, at a smooth 60 FPS.

Commits

As always, this is not a complete list of PRs or commits, nor does it necessarily list every single thing done by a given PR. For a full list, see PRs merged in September. Many of the unmentioned PRs are simply small updates, fixes, and other small improvements, sometimes only to the GUI.

Nekotekina

3503 – Implements basic networking support and also corrects some HLE video decoder bugs found when running the Crunchyroll program.

3391 – Miscellaneous bug fixes in various areas such as trophies (not implemented in the frontend yet) and the file system.

3411 – Fixes various bugs in the file system module and memory mapper.

3468 – Fixes various bugs in the virtual filesystem used by the emulator, and in library loading when automatic and manual loading is combined.

3454 – A significant PR with fixes in various areas such as library loading, file system, and logging. Among many things this fixes the launcher in the Devil May Cry HD Collection and allows one to play the game as Lucia which didn’t work before. Therefore the entire collection is now playable. This also fixes the launcher in Ridge Racer 7 taking it ingame for the first time.

kd-11

3475 – Workarounds for driver bugs and fixes for random crashes and hanging experienced by some people.

3488 – Improved frame rate and frame times with Vulkan and fixes some GPU texture scaling crashes.

3458 – A significant number of graphics emulation improvements detailed closely above in Major Improvements

flash-fire

3420 – This PR from flash-fire fixed a lot of Unity games causing RPCS3 to crash (RPCS3 stopped responding) when launching with PPU LLVM. While they don’t progress much further it’s definitely still good progress.

Farseer2

2894 – Various fixes (Trophy, Gcm corrections, stack alloc) The Trophy fixes allowed Dragon Ball Raging Blast 2, Kung Fu Panda 2, Budokai HD Collection and more to progress further, some even go ingame now.

scribam

3426 – Scribam begun working on adding a new column to the RPCS3 user interface that can display the status for your games. This will be completely optional, users who do not like the idea of RPCS3 sending your game list information to the server have nothing to worry about.

Patreon

The $3000/month goal on Patreon was reached in the middle of September, and thanks to extensive media coverage caused by a certain event it was even surpassed! This practically means that kd-11 can work on RPCS3 full time, and he is currently in the transition of doing so. The extra time he has been able to spend on RPCS3 in September already shows as mentioned earlier with his major graphics emulation improvements, and also as seen below in the upcoming developments.

Compatibility Database & Future Wiki

Ani restructured the way the Compatibility Database works as a preparation for Game IDs batching. This basically means that different Game IDs that represent the same Game Titles will be displayed together as if they were one game entry – one for each type of Game Media (Disc and Digital).

So, for example, instead of having 4 entries for BLES00199, BLUS30089, NPEB00393, NPUB30451 – which represent Assassin’s Creed European and US Disc and Digital copies, we will have 2 entries, 1 for Assassin’s Creed Disc versions and 1 for Assassin’s Creed Digital versions. Every Game ID will still have its own thread for discussion on the forums and every list entry will still link to all different region threads.

Currently the Compatibility Database has approximately 86% unique title entries on Digital games and 81% on Disc games, meaning the amount of titles that have more than one of their IDs (under the same media) listed is considerable.

After the work on updating the Compatibility List is done, work on a Wiki will start as mentioned previously on our forums’ opening announcement post. Firstly the wiki will list information for RPCS3 general usage and our Playable games. As the wiki expands, further information will be added.

Official RPCS3 Website Exits Beta After Redesign

DAGINATSUKO redesigned rpcs3.net from scratch back in January replacing an old wordpress based website that for some insane reason literally used the comic sans font. Take a look around, check out the different pages such as the front page and the about page. Also try out the dark mode by pressing the top right button in the navigation bar.

rpcs3.net is open source and you can find the source code here.

Upcoming

Some very exciting developments have happened behind the scenes for RPCS3. As teased in an announcement days ago, high resolution rendering is coming soon, along with anisotropic filtering! Seen below is a comparison of how the main character in Ni no Kuni looks like in the original 720p and 1440p as well as a distant sign in GTA San Andreas. There is also a screenshot of Demon’s Souls running at 3840×2160 with 16x anisotropic filtering. High resolution rendering should probably be available in one or two weeks. But just around the corner, even bigger news is developing…

Click for full quality PNG.

Closing Words

If you like in-depth technical reports, early access to information, or you simply want to contribute, consider becoming a patron! All donations are greatly appreciated. As of last month, RPCS3 now has two full-time coders that greatly benefit from the continued support of over 600 generous patrons.

Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by ssshadow, Asinine, Talkashie, and Ani.