Progress Report: January 2019

Welcome to the January 2019 progress report! This month saw significant improvements to the core components of RPCS3 such as the introduction of multithreaded cache compilation for the SPU LLVM recompiler, reimplementation of the graphics framebuffer management, overhaul of the audio backend and much more. We also saw RPCS3’s version bump to 0.0.6 to better showcase the state of the emulator.

In addition to the following report, further details of Nekotekina and kd-11’s work during January and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from Nekotekina (2019-01-01)
Status update from kd-11 (2019-01-02)
Status update from kd-11 (2019-01-27)

Table of Contents

Major Improvements
Games
Other Improvements
Conclusion

Over at our forums, a few user and moderators have come together to acquire and test titles that have not been tested recently. Our developers have also been hard at work debugging niche issues that seem to prevent a few titles from progressing ingame. The results of their targeted efforts gives us veracious compatibility statistics from which we can see a big decrease in Intro and Loadable categories and a corresponding increase to Playable and Ingame categories.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (January 2019)

On Git statistics, there have been 8179 lines of code added and 5073 removed through 36 pull requests by 9 authors.
Continue reading Progress Report: January 2019

Progress Report: December 2018

Welcome to our (2018) December Progress Report! We are extremely happy to finally push this report out to you guys after the many delays, for which we tried to make up with other kinds of updates along the way. We hope that you’ll enjoy our deep dive about the last month of the past year, and find its stories equally as intriguing.

December has been an absolute whirlwind for RPCS3 with improvements made to various parts of the emulator. Nekotekina greatly improved performance with the introduction of Approximate xfloat while kd-11 fixed a wide variety of graphical issues affecting multiple AAA titles. Apart from our full-time developers, our regular contributors also made a plethora of improvements to the texture cache predictor, cellCamera emulation, controller/mouse support, GUI and tons of bug fixes. We’ve tried to detail as many as we possibly could in this report. So without further ado, let get into it!

In addition to the following report, further details about Nekotekina and kd-11’s work during December and their upcoming contributions can be found in their weekly reports on Patreon. December’s Patreon reports were:

Status update from kd-11 (2018-12-10)

Table of Contents

Major Improvements
Games
Other Improvements
Upcoming
Conclusion

This month saw improvements to various aspects of the emulator and the compatibility list is no exception. At the start of 2018, the compatibility list was revamped to allow us to merge multiple game IDs for the same game into one single entry. This helped reduce duplication of entries and enhance the quality of the statistics we provide. While we were able to merge IDs of different regions together, due to the way the compatibility list database was structured, titles with multiple IDs from the same region could not be merged and hence were shown as separate entries. Not wanting to leave the job half done, AniLeo refactored the compatibility list once again this month to now allow us to merge IDs from the same region as well!

With this change, we can eliminate unnecessary duplication entirely and significantly improve the quality of our statistics to properly represent the amazing progress made with the emulator. If you take a look at the metrics for this month, you will notice that the number of games has decreased across all categories. This is solely due to the above mentioned revamp of the compatibility list. However, if you take a look at the compatibility history page, we can see that the Playable category has increased by over 15 titles and Ingame category has increase by over 30 new titles. The nothing category also reduced by one, bringing the total to just 3 titles!

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (December 2018)

On Git statistics, there have been 4,802 lines of code added and 2,530 removed through 105 commits by 18 authors.
Continue reading Progress Report: December 2018

Progress Report: November 2018

Welcome to November’s Progress Report! As 2018 now comes to close, we hope you’re enjoying the holiday season. In this report, we will be detailing kd-11’s work on rewriting the FIFO and draw call processing which provides a noticeable improvement in performance in quite a few games. This month also saw the return of another long-time contributor, GalCiv whose contributions fixed a large number of regressions in multiple games. We also saw exclusive titles such as Gran Turismo 5 and MLB: The Show 16 go ingame for the first time on RPCS3. We’ve got lots to share so let’s jump straight into it!

In addition to the following report, further details of Nekotekina and kd-11’s work during November and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from Nekotekina (2018-11-04, part 2)
Status update from kd-11 (2018-11-08)
Status update from kd-11 (2018-11-24)
Status update from Nekotekina (2018-11-30)

Table of Contents

Major Improvements
Games
Other Improvements
Upcoming
Conclusion

For the first time, the percentage of games in the Ingame and Playable categories have risen above 80%. This marks yet another milestone in game compatibility for RPCS3. As the accuracy and performance improves, we will see this ratio improve even further! Thanks to the intensive testing done by our testers this month, 95 games were moved out of Intro and Loadable categories. Also, multiple duplicate titles across all categories were identified and merged, improving the overall accuracy of the compatibility list. Finally, the elusive Nothing category has dropped to just 4 games. For a more detailed look, you can view the compatibility history page to see exactly which games had their status changed this month.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (November 2018)

On Git statistics, there have been 4,229 lines of code added and 1,843 removed through 30 pull requests by 11 authors.
Continue reading Progress Report: November 2018

Progress Report: October 2018

Welcome to October’s Progress Report! Thankfully, there have been no major delays in publishing this time around, and we hope to continue gradually closing the gap even further in the future.

On the other hand, since both of our main developers were busy this month doing heavy refactors and rewrites in the background, not all of the changes are ready to be showed off just yet. While Nekotekina’s current efforts aren’t about yielding direct and spectacular changes to begin with, we did do a write-up about kd’s changes and another graphics related PR from ruipin here in the Upcoming section below.

This isn’t to say that October was without any changes, as one entry of a cult-classic and previously PlayStation exclusive AAA series, Yakuza, just hit playable this month! For this, and more, let’s dive in and see what happened during the month!

In addition to the following report, further details of Nekotekina and kd-11’s work during October and their upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2018-10-07)
Status update from kd-11 (2018-10-21)
Status update from Nekotekina (2018-10-28, part 1)

Table of Contents

Major Improvements
Games
Other Improvements
Upcoming
Conclusion

Looking at the compatibility database, we can see the numbers continue to move in the right direction at a slow, but steady pace. Around 60 new games have been added to the Ingame and Playable categories, and several has transitioned up between those. Also, with the consistent bug fixes and accuracy improvements, our developers and contributors continue to chip away the Intro and Loadable categories, as well as improve the games of other categories. For further details, take a look at the compatibility history page to see which games in particular had their status’ changed during the month.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (October 2018)

On Git statistics, there have been 4,128 lines of code added and 2,583 removed through 30 pull requests by 8 authors.
Continue reading Progress Report: October 2018

Progress Report: September 2018

Welcome to September’s progress report! Firstly, we would like to apologize for the delay. Our progress reports are written by voluntary writers and sadly most of them were unavailable to contribute this month. However, there is a silver lining here. The additional time we had gave us a unique opportunity to convert this month’s progress report into a technical exposition hybrid!

We’ll be featuring a deep dive into the inner workings of the texture cache in RPCS3, and how it was improved thanks to the contributions of ruipin and Nekotekina. We will also uncover the wide variety of improvements that kd-11 made and showcase some massive improvements to various AAA titles. Without further ado, let’s jump straight into September’s irregular progress report!

In addition to the following report, further details about Nekotekina’s and kd-11’s work during September and about their upcoming contributions can be found in their weekly reports on Patreon. The month’s Patreon reports were:

Status update from kd-11 (2018-09-10)
Status update from Nekotekina (2018-09-17)
Status update from kd-11 (2018-09-23)

Table of Contents

Major Improvements
Games
Other Improvements
Conclusion

In the compatibility database statistics, we can see all the numbers moving further in the right direction. The Ingame category has breached the 1300 games barrier while Playable continues to slowly raise due to the amount of time it takes to make a playable compatibility report. Intro also saw a decent reduction as a result of recent improvements and lots of testers making compatibility reports. For more details, take a look at the compatibility history page, to see which games in particular had their status’ changed during the month.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (September 2018)

On Git statistics, there have been 10,178 lines of code added and 6,751 removed through 53 pull requests by 12 authors.
Continue reading Progress Report: September 2018

Progress Report: August 2018

August has been an amazing month for RPCS3 as we crossed multiple new milestones. This month saw massive performance improvements to many AAA titles, accuracy and performance enhancements to SPU LLVM, support for C++ 2017, laying the foundation for macOS support and much more!

In addition to the following report, further details of Nekotekina and kd-11’s work during August and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2018-08-09)
AAA PR Update
Status update from kd-11 (2018-08-21)
Status update from Nekotekina (2018-08-30)

Table of Contents

Major Improvements
Games
Commits
Upcoming
Conclusion

The Playable category has finally crossed 1,000 titles milestone! Considering that this time last year, the Playable category was only a little over 400, it truly demonstrates the amazing pace of development. For all other categories, we can see the metrics moving in the right direction with the elusive Nothing category dropping by 1, with only 5 games remaining in it. For a more detailed look, you can view the compatibility history page to see exactly which games had their status changed this month.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (August 2018)

On Git statistics, there have been 7,086 lines of code added and 4,298 removed through 127 commits by 21 authors.
Continue reading Progress Report: August 2018

Progress Report: July 2018

This month has seen major improvements to the SPU LLVM recompiler and RSX emulation as well as implementation of new features such as a user manager and a refreshed UI design, among many other improvements and additions.

In addition to the following report, further details of Nekotekina and kd-11’s work during July and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2018-07-05)
Status update from Nekotekina (2018-07-12)
Status update from kd-11 (2018-07-21)
Status update from Nekotekina (2018-07-30)

Table of Contents

Major Improvements
Games
Commits
Conclusion

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (July 2018)

On Git statistics, there have been 8 047 lines of code added and 4 671 removed through 135 commits by 12 authors.
Continue reading Progress Report: July 2018

Eliminating Stutter with Asynchronous Shader Implementation!

Background

Shader compilation stutter is nothing new to most emulator users, especially on RPCS3. However it is worth clearing up some misconceptions that go around regarding how RPCS3 shaders work. I’ll try to quickly go over the history of shader compilation on RPCS3 and hopefully explain why the shader compilation stutter appeared and why some people believe RPCS3 did not have shaders before.

Shader complexity and custom vertex fetch

In early 2017, I embarked on a task to remove the very expensive vertex preprocessing step from the CPU side of RPCS3. This basically meant implementing all those custom vertex types and vertex reading techniques to the vertex shader and providing only raw memory view that the ps3 hardware would be viewing. This greatly improved RPCS3 performance, more than tenfold in some applications. This change is what made RPCS3 usable for playing real commercial games with playable framerates without needing HEDT system. However, the new fetch technique increased the size of the vertex shader and added a complex function to extract vertex data from the memory block. This made the graphics drivers take very long to link the programs, even without optimizations, likely due to use of vector indexing, switch blocks and loops with dynamic exits. Extra operations including bitshifts and masking were also needed to decode the vertex layout block. The code runs very fast, but the linking step is very slow. A shader cache system already existed before and if you ran an area for the first time, there was slight microstuttering that some users did not notice; its this stutter that got much worse. The solution to this: preload the shaders so that you don’t need to compile them next time. This lead to the infamous “Loading Pipeline Object…” screen and the “Compiling shaders…” notification.

Challenges

There are several challenges to tackling RSX shaders. First, the RSX is not a unified architecture like most programmers are used to today. It has separate vertex and fragment pipelines, both with their own separate ISA. They are also very limited and larger programs or more complex programs can result in very messy binaries. One of the largest problems is that the bytecode itself does not contain all the information required to run the program, extra configuration is configured via registers as draw calls are passed in. A good example is that the TEX instruction does not differentiate between texture types, but a texture configuration register exists that allows using the same program to read 1D, 2D, 3D, or CUBE textures as well as their shadow comparison variants. This means you can only know the generated shader once the texture register has been set up. There are other examples of things like these that make it so that you need the game to set up the program environment before the program itself is compilable.
Continue reading Eliminating Stutter with Asynchronous Shader Implementation!

Progress Report: June 2018

In addition to the following report, further details of Nekotekina and kd-11’s work during June and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2018-06-05)
Status update from Nekotekina (2018-06-12)
Status update from kd-11 (2018-06-20)
Status update from Nekotekina (2018-06-27)

Table of Contents

Major Improvements
Games
Commits
Upcoming
Conclusion

On compatibility database statistics, we can see a big decrease in Loadable games due to several recent changes targeting issues that kept those games from progressing further. The Nothing category also went down by 2 games, with only 6 games remaining in it.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (June 2018)

On Git statistics, there have been 16 605 lines of code added and 10 558 removed through 135 commits by 21 authors.
Continue reading Progress Report: June 2018

Progress Report: May 2018

RPCS3 continues to see major improvements over the month of May, with Nekotekina implementing SPU LLVM (still WiP) and kd-11 continuing with improvements to RPCS3’s RSX emulation. More major AAA exclusives have also started to go ingame for the first time! We also saw new contributors join in and make much appreciated improvements to RPCS3.

In addition to the following report, further details of Nekotekina and kd-11’s work during May and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2018-05-05)
Status update from Nekotekina (2018-05-14)
Status update from kd-11 (2018-05-20)
Status update from Nekotekina (2018-05-29)

Table of Contents

Major Improvements
Games
Commits
Conclusion

The Nothing category is now at an all-time low, reducing by just over half, work is already being done to make this category shrink even further! The Playable category has had another nice increase, as more Playable titles were found. Furthermore, many entries for the same Game Media on the list were merged (due to new reports for different regions being submitted these past months), so the overall game count has decreased, even though there were new unique submissions. For a more detailed look, you can view the compatibility history page to see exactly which games had their status changed this month.

Game Compatibility: Game Status
Game Compatibility: Monthly Improvements (May 2018)

On Git statistics, 23 166 lines of code were added and 6 732 were removed by 17 authors.
Continue reading Progress Report: May 2018