Nearly six months after the Patreon launched the RPCS3 team have finally improved RPCS3 on Linux to the point where it has reached compatibility and stability parity with Windows. Thanks to the hard work by hcorion we can finally start to provide pre-compiled binaries in the form of AppImages for easy installation on your favorite distribution.
What Was the Issue?
There were a lot of problems. Back in January, quite literally nothing was working. RPCS3 would crash instantly upon booting any game, if the program itself would even start at all. Moreover, additional functionality like the debugger, framerate counter, and firmware installer were completely broken too. While many of these auxiliary issues were quickly identified and fixed, the fact of the matter was that almost every game would hang after running for a few seconds. This turned out to be much more difficult to fix. This was caused by several different bugs in thread synchronization which were fixed continuously in the past few months. Finally, one last relatively small commit in early April fixed the last bug and suddenly RPCS3 on Linux went from basically nothing to Demon’s Souls and Persona 5. Or so we though, but we quickly found out that the LLVM recompiler was completely broken for a lot of users who just got completely nonsense errors. We encountered strange and esoteric bugs and oddities about LLVM and how various Linux software, including the Mesa drivers, were using it. These problems made RPCS3 unusable for a lot of people. A lot of false flags and red herrings later the bugs were fixed not by changing any code, but by using rare compiler flags. RPCS3 on Linux is now working as intended for everyone, including AMD and Intel graphics users with modern Mesa. Even Vulkan with Mesa is now working!
PlayStation 3 Games on Linux
Below are some popular PlayStation 3 games showcased running on Linux. Performance is about the same as on Windows, perhaps even a few percent better in some very intensive games like Persona 5. But take note: These images were either captured on a laptop with a very old i7-2670qm CPU, or a fast desktop with a i7-4770 CPU.
Last week the Persona 5 video seen below was posted showing how RPCS3 had been improved to fix various emulation issues with the game. The issue where Persona 5 had broken bloom and depth of field, making various parts of it have incorrect colors and also look blurry, was fixed. Additionally performance had been improved thanks to the reworked PPU recompiler a few days earlier.
While game certainly looks playable in the video it really wasn’t until today. Two critical issues persisted: The game would crash every time when starting a battle if the frame rate was higher than about 10-15 fps, and without opening the RPCS3 debugger and pausing a few special threads performance would suffer. But these problems are now fixed, and using the work in progress build linked below Persona 5 is playable.
Is Persona 5 Really Playable?
Yes, a few people have already beaten the entire 100+ hours long game in RPCS3 so essentially every issue is known to the developers, and all of the critical problems have been fixed. As the game looks more or less fine, performs more or less fine, and is provably beatable now without any strange hacks and workarounds we classify it as playable, but not perfect. One important aspect of this is that the Persona 5 engine is “framerate unlocked” so to speak, that is the game runs at full speed at almost any frame rate, like most PC games do for example. The actual interval where it runs at full speed is 15 – 60 fps, and for example an overclocked Haswell i5 can expect to play the game at about 10 – 20 fps in school and around Tokyo, and at about 20 – 30 fps in dungeons and battles.
A RPCS3 screenshot showing that the game has been completed. Take note that the displayed play time is lower than the actual play time as the game slows down below 15 fps.
Fixing the battle crash
The game would start to crash when engaging in a battle if the frame rate was above about 10 – 15 fps or so. As frame rate increased so did the probability of the crash happening, and at 30 fps it was definitive. When investigating this it was discovered that the crash was happening in the graphics renderer thread because it was told to process a frame with a null pointer instead of a fragment shader. Combined with the previous knowledge that the probability of the crash increases with the frame rate it sounds like a sync issue in RPCS3, but actually it is probably a game engine bug. Recording the raw output of a real PS3 and stepping through it frame by frame we notice something interesting:
Take a look at what is labeled as frame two and three. Every frame before and after these are unique, but these two are duplicated. These duplicated frames occur exactly in the battle transition where RPCS3 was crashing. Could it be that the bad frame is actually bad on a real PS3 too, and the real graphics driver simply ignores it? Seems like it.
Fixing the need for thread pausing
This was a very interesting way of increasing performance in Persona 5. Like most games there are several threads running at the same time (around 20 in this case) doing various things like audio decoding, file loading, etc. Three to four out of six “cellSpursKernel” threads, running on the SPU part of the Cell CPU, could actually be paused when playing the game and for some counterintuitive reason the game would keep running just fine. Not only that, frame rate would increase and most noticeably lag spikes would be eliminated. The question can be asked: But surely if the threads can be paused indefinitely like that they can’t be doing anything important, and if they are not doing anything why is there such a noticeable performance increase?
It turns out that they are doing important work, but as a performance optimization on the real hardware several of these threads would be doing the same kind of work on the same data at the same time. On real hardware the first thread to start would get to do the work, the other threads would either do nothing or grab the next piece of data so to speak. However this high level of concurrency was not really happening in RPCS3, instead they would deadlock each other and in general just slow things down compared to letting one thread do everything. Hence pausing almost all of the threads made the game faster. The fix is to simply detect when multiple cellSpursKernel threads are about to do the same thing and stop them if they do. This fix could also speed up other games with the same design, for example Red Dead Redemption sees a small performance improvement.
Persona 5: from Nothing to Playable in Half a Year
From the initial release of the game in Japan in september of 2016 up until the middle of January in 2017 the game would instantly crash upon starting it with a stack overflow exception. Some fixes to how fibers are handled later and it would start up and with some luck go ingame and display one or two very broken looking frames at 1 fps before hanging forever. Here is the world’s first screenshot of Persona 5 going ingame in RPCS3:
January was also the month when the RPCS3 Patreon launched, and the quick success of it allowed Nekotekina to work on RPCS3 full time. Graphics developer kd-11 also unofficially joined soon thereafter, and from there everything was set in motion. A day before the official English release of the game in April of 2017 we got an i7-5930K to record a video of the first hour or so of the game. This got a lot of attention, close to 200,000 views and articles in various gaming sites (and a look of disapproval tweet from Atlus).
Various fixes and a partially redone recompiler later we are here today where the game can finally be classified as playable on a hardware level such as an overclocked i5-4690K.
Download the Build
For now some of the changes required for Persona 5 are work in progress, therefore download and use this build for now. This needs to be overwritten to an existing RPCS3 setup, download that here, and please also read the quickstart guide for critical information on how to install the PS3 firmware in RPCS3 and how to dump the game disc or PSN download, as well as other information. Optimal settings for Persona 5 are the recommended settings in general: PPU and SPU recompiler, Vulkan renderer.
While Persona 5 is now playable this is only the beginning. RPCS3 still needs a lot of work in order to run games like Red Dead Redemption, Metal Gear Solid 4, and The Last of Us. If you want to help out Nekotekina and kd-11 and make RPCS3 progress even faster you can check out the Patreon page here.
Also, come check out our YouTube channel and Discord to ask for help with using RPCS3 and to stay up-to-date with any big news.
Another month another colossal RPCS3 progress record. But first things first, take a look at this beautiful introduction to RPCS3 video that reznoire made:
Moving on to some statistics the numbers alone this month are quite impressive, and that is not even counting games that improved from emulation improvements that are still work in progress.
Looking at the GitHub statistics 16 authors have pushed 140 commits to the master branch. Here 353 files have changed and there have been 19,729 additions and 17,660 deletions. These numbers are much larger than normal, and several exciting changes are behind it. This month RPCS3 transitioned away from the previous GUI toolkit Wx to Qt which in turn also lead to several user facing interface improvements. This separate report goes into significantly greater detail regarding what work has already been done on the GUI, and what work is planned to be done in the future. On a lower level, Nekotekina reworked the entire PPU LLVM recompiler to greatly increase compatibility of it. kd-11 fixed various graphical issues affecting hundreds of games, most notably broken shadows and depth of field in various titles. Moreover kd-11 also enabled Vulkan on Linux and pushed support for it to the master branch. Nekotekina and Numan also worked on .sprx relocations in general and in the LLVM recompiler which in simple terms is a compatibility and performance improvement.
The talented developer Jarves is working on a very important core improvement. It is not yet finished or merged to the master branch, but as work on this has been going on in public, and a lot of people have already tested it and submitted results this report will cover the work anyway. Jarves is working on “LLE gcm” which is huge and here is why:
LLE stands for Low Level Emulation. In RPCS3 this means that a PS3 operating system module is run directly as it is via lower level emulation methods. This is great for compatibility because it doesn’t involve much guesswork and is very accurate. As far as the games are concerned, they are working with the exact same operating system methods with the exact same implementations here as on a real PS3.
gcm or “graphics command management” is the part of the PS3 operating system responsible for creating and managing various graphics commands, including everything from how to set up vsync to graphics memory allocation.
LLE gcm is huge. A very core part of emulating games is now done exactly the same as on on a real PS3, and the result is amazing. For example Red Dead Redemption no longer hangs almost instantly. The Last of Us doesn’t crash immediately and actually shows a loading screen and goes through a lot of initialisation successfully. Persona 5 no longer randomly crashes. Various high end games that did nothing before are now starting to boot, for example the Yakuza series where Yakuza 3 went from nothing to ingame, and Yakuza 4 and 5 went from nothing to loading screens. One could go on and on about what a substantial compatibility improvement LLE gcm is (and I will). Now combine this with the greatly improved LLVM recompiler and over 40 commits of graphics fixes and you have a month of quite insane progress. This would be a great segway into looking at improved games and such, but Jarves did one other important contribution that deserves its own section.
June has been an exciting month for RPCS3. Quite a few new features have been added to the emulator, with a healthy focus on the somewhat outdated GUI. This post will break down some of changes brought by Qt, as well as new features introduced since the transition. This mini progress report will only focus on GUI changes. The main progress report to be published tomorrow will cover the rest.
First up is, obviously, the actual transition from the user interface toolkit WxWidgets to Qt. This transition has been a long and time consuming process, but it added a host of new functionality, both visible and not so visible. A non-exhaustive list of the transition pull requests, the main of which can be found here is:
– Appveyor and Travis now build with Qt project, thanks to hcorion. This switches the nightly builds, and eventually the linux builds, to use the Qt interface.
– Made some design changes to the GUI, such as progress bars and taskbars now showing percent completion and slight improvements to the Vulkan/DX12 adapter selection box to make it easier to use.
– Added support for layouts, which allows you to move your docked widgets as you please.
– Added support for booting games in fullscreen.
– Added a Welcome screen to point new users to the Quickstart setup guide.
– Support for themes! An example can be found further down.
As with all major changes, this caused a few hiccups, but they are being worked on as they are found, and issues can be reported here.
This transition to Qt brought a wave of GUI improvements with it. First up was a recent games list, implemented in #2843 and #2847. This saves the last nine games launched in an easy to use list that has hotkeys, so that those with large libraries, or those using Blu-ray disc games, can easily launch their favorites. The list can be frozen, though items cannot yet be pinned.
Second was a new viewing mode and a tool bar. The game list can now be viewed as a simple grid:
May was a very eventful month for RPCS3 as we saw significant core and performance improvements. The goal of this progress report is to highlight some of the more notable or interesting developments of the project. The report will start by showcasing a selection of games that were improved in one way or another. Thereafter we will summarize what work each contributor did this month.
kd-11 joined Nekotekina’s Patreon
You can read about this in more detail here.
The short version is that kd-11 is an extremely talented graphics developer who has helped RPCS3 since January of 2016. Without his work RPCS3 would not be here today. It goes the other way around too of course, without the significant core accuracy and performance improvements by Nekotekina we would not be here today either. Therefore it make sense for two great minds to join forces. With the generous Patreon support kd-11 will be able to acquire new hardware for development and testing. On the short term purchasing list is a modern NVIDIA GPU, a modern AMD GPU, and possibly a Skylake+ laptop of some kind. This would allow kd-11 to fix specific issues on these platforms. Not many people use RPCS3 with integrated Intel graphics but they are technically fast enough (for now at least). The problems lie in very very buggy Intel graphics drivers which is why it would help to have direct access to it.
Check out this video below, it highlights some games that saw performance improvements from “the secret build” which was a huge general graphics rework, with focus in particular on Vulkan.
Veteran RPCS3 graphics developer kd-11 is joining Nekotekina’s Patreon. kd-11 has worked on RPCS3 since January 2016 and has since then fixed countless bugs in all graphics renderers and implemented a lot of missing functionality. Without his dedicated work, RPCS3 would certainly not be where it is today.
As you know lead core developer Nekotekina, who has worked on RPCS3 since 2013, launched his highly successful Patreon in January this year. Thanks to the overwhelming amount of support, he has been able to work on RPCS3 full time for 5 months now and the progress he has done is staggering. The RPCS3 team has discussed the best course of action for the road forward and have decided that the time is now right for kd-11 to join Nekotekina’s Patreon in order for him to obtain hardware necessary for development and testing.
As kd-11 works on the graphics side of things, especially the renderers using OpenGL, Vulkan, and D3D12 (Direct3D 12), modern hardware is required for development and testing. Ideally, he needs a modern AMD, Nvidia, and Intel GPU to easier identify bugs specific to one platform. Furthermore, kd-11 works as a consultant and therefore has the possibility to accept or turn down certain jobs. If support on Patreon reaches a certain level, he may be able to spend more time on the project.
April was a very eventful month for RPCS3. The goal of this progress report is to highlight some of the more notable or interesting developments of the project. The report will start by showcasing a selection of games that were improved in one way or another. Thereafter we will summarize what work each contributor did this month.
Since the last progress report, approximately 18 authors have made 104 commits, added 9,621 new lines of code and deleted 1,904 lines of code.
This is the last real PlayStation 3 game and it also happens to be highly popular. Let us take a look at a few things.
People often ask what kind of CPU is needed to run this game. The answer is that no CPU is truly fast enough right now, but if you enjoy playing games at 10 FPS or so, then feel free to get a CPU with a lot physical cores. See the screenshot below for the motivation behind this statement. Of course in the future when RPCS3 performance is improved such an extreme CPU likely will not be required.
March 2017 beat the previous record set by February 2017 as the most eventful month in the history of the project. So much happened that even this colossal progress report can only begin to scratch the surface.
First of all, lead RPCS3 developer Nekotekina reached the $1000 goal on Patreon, securing his long time commitment to work on RPCS3 full time. This was the direct result of the massive amount of attention the project got when two popular games were drastically improved this month. First, Demon’s Souls went from crashing in zero seconds to going ingame and almost being playable. Second, the cult classic Catherine received significant performance improvements and now runs with practically perfect graphics and performance. The two videos of these games received over 200 000 views on the official RPCS3 YouTube channel. Even the famous YouTube personality, TotalBiscuit was impressed with the progress, and the killing of the notorious Demon’s Souls boss Vanguard in RPCS3.
Of course the hundreds of videos posted by the community on YouTube, different forum posts, Reddit submissions, and so on, have contributed greatly to the massive growth of the RPCS3 community. In fact, new developer Inviuz aka Numan was one of these people who recently discovered RPCS3, spent a lot of hours reading the code and debugging Demon’s Souls, and finally getting it to boot for the first time. Oh and Red Dead Redemption also went to the main menu thanks to this, and some graphics improvements by kd-11. This showcases one of the most important strengths of RPCS3 being free and open-source software and it is very likely that more and more people will join the project in the future and contribute changes both big and small.
Since the last progress report, approximately 17 authors have made 115 commits with 6,831 lines of code added, and 4,471 lines of code deleted. And some significant improvements are still in the pipeline but have yet to be merged.
The progress report is mainly split into three different parts. First we will take a closer look at what each pull request this month did, and show a few practical examples. Thereafter we will take a look at a selection of some interesting games that were improved this month, though this is just a small slice of the hundreds upon hundreds of games that received small or significant improvements this month. Lastly we will take a look at some of the upcoming changes before rounding of this monthly progress report.
It finally happened! The PlayStation 3 emulator RPCS3 has launched one of the most iconic games on the platform: Demon’s Souls. Are you curious to know how it all happened? What the developers went through? Welcome to this first of hopefully many to come developer logs where we will now tell you everything.
It all began in our Discord channel where a user going by the nickname, Numan approached us with some interesting news. He claimed to have conducted a detailed investigation of what Demon’s Souls was trying to do when it was crashing one second after booting. He drew the conclusion that the game was trying to allocate memory pages of different sizes, access levels, and different attributes. When the game then tried to access a specific memory address, it crashed with an access violation. Consequently, these discoveries lead to a more accurate implementation of the function sys_memory_get_page_attribute() which was responsible for this particular crash. Suddenly the game booted up in RPCS3 for the first time ever and showed a loading animation!
February 2017 was one of the most eventful months in RPCS3 history. Earlier this month we reached the first Patreon goal of $500, thus ensuring that Nekotekina can continue to work on RPCS3 full time for the time being. A total of 17 authors have pushed 127 commits to the master branch, with 9882 lines of coded added and 6575 lines of code deleted. This represents several hundred, if not thousands hours of work on the project, and it really shows.
In this progress report we will take a look at what each person has been working on for the past month, and highlight some of the more noteworthy changes. This is however far from a complete list of contributions and improvements. Several people in the community have tested hundreds of games, reported several issues, made YouTube videos and supported people on Discord. Nekotekina, kd-11, and the rest of the RPCS3 team would like to thank everyone for their contributions to the project.