WaitTOF() does not busy loop? what other API's can you use to wait for VSync?
Set INTERRUPT=NO in the monitor file's tooltypes. That disables using interrupts for vblank, and calls to WaitTOF(), WaitBOVP(), and ChangeScreenBuffer() will internally use the busy-waiting fallback.
Quote:
Just checking, but there is no v5 driver compatible with Pegasos2, correct?
Ugh! Yes. I forgot about that (I don't buy drivers that I write ).
Quote:
The RX cards don’t work under qemu as the driver gets stuck in a loop so under qemu at least we’re limited to HD cards.
Not necessarily. Geennaam manged to stop the infinite loop using the script found under this video. If that doesn't work, then try "connecting" the graphics card pass-through to PCI bus 1 instead of 0.
Not necessarily. Geennaam manged to stop he infinite loop using the script found under this video. If that doesn't work, then try "connecting" the graphics card pass-through to PCI bus 1 instead of 0.
I'm already on bus 1 but I will take a look at that video - thanks.
I know nothing about resizable BARs so can't comment on that
Resizable BARs is a feature that newer graphics cards have. It's accessed via a PCI capability in config space, and allows the VRAM BAR to be resized so that more of VRAM is directly visible by the CPU. Graphics cards typically limit the VRAM BAR to 256MB by default because otherwise 32-bit OSes might run out of address space for other stuff.
I will only post my full command line once at the start. For all other tests I'll only post any change (for example when I change the bus or address of the GFX passthrough):
In each case "INT's" on or off refers to "INTERRUPT=Yes|No" in the monitor file.
OK - all these tests are with R9 270X card
1. CMD Line as above. Current master. No patches. INT's = Yes. Doesn't even get past the startup sound before the system halts. The mouse can be moved until you click something but that's it.
2. CMD Line as above. Current master. No patches. INT's = No. Works. Every other boot I seem to get no sound (unrelated?). Generally feels sluggish everywhere. Network download speed feels slow, display is sluggish. 02-QuakeMap test, around 30fps
3. CMD Line as above. Current master plus OSDN patched master. INT's = Yes. Same as before can't get into workbench before the system hangs.
4. CMD Line as above. Current master plus OSDN patched master. INT's = No. Everything considerably quicker. Internet download speed way quicker for example. I really should measure! 02-QuakeMap test around the same 30fps. SuperTux platformer up from 3fps to 28fps. Sound still randomnly doesn't work.
5. CMD Line as above. Current master plus original mbox patch. INT's = YES. Overall not as snappy as 4 but better than 2. Internet download speed bears this out, it's up near the speed of 4 but not quite there. No hangs or crashes. QuakeMap is down to 10fps. SuperTux is at 7fps.
6. CMD Line as above. Current master plus original mbox patch. INT's = No. Basically the same as 4 but I don't seem to randmonly get no sound. So I'd say this is the sweet spot at the moment but there's little in it between your new patch and this one with GPU interrupts off.
I haven't run any of these tests with the card on bus 0 yet. I think this is all too vague and not desperately helpful. Apart from whether there are any hangs or not.
I need to see if there are some better tests I can do.
@MartinW Everything with the "INTERRUPT=No" ToolType set isn't really usable, it's just a workaround to be able to use the gfx card at all - but it will definitely cause problems with other hardware/drivers, especially on the Pegasos2 with everything, except for the VIA PATA controller, using the same interrupt #25. All other AmigaOS 4.x systems use separate interrupts for each PCI slot instead. The only usable variant is therefore your 5., not 6.
Honestly, I don't really know what's going on now!
I put the RX550 card in and re-ran the device isolation script. I was already using it but had swapped cards since. I thought it was only enabling IOMMU and binding the device to the VFIO driver.
I can boot to OS4 workbench with the RX card which is progress but only with interrupts disabled and workbench is terribly laggy. When you do eventually manage to get to SuperTux it runs pretty well!
But this is when it gets weird. The first time you run Qemu you will get an error when the firmware tries to load the BIOS. This is nothing new it's happened all along. On the second time I run qemu I noticed I didn't get that same error. The error had a smaller message. Shortly after I got a hard host crash / reboot. That's unexpected.
Subsequently I ran QEMU once. OK. Ran it again but made a typing mistake so quit before booting the OS. This time on 3rd run I didn't get an error at all from the firmware and it just loaded to the OK prompt. I again quit. The 4th and subsequent times I attempted to run qemu it gave me a resource error and said it couldn't allocate PCI space and I had to reboot.
I have no idea what's going on here!
[edit] That was all with the RX card on pcie.0 If I instead specify pcie.1 like I have been doing with the HD cards then I am back to the inifinite loop of
RadeonRX (5): RadeonRX (5): Cannot print bridge configuration for PCI:0.0,0, because it is not a bridge device.
RadeonRX (2): Cannot enable blind prefetch for PCI:0.0,0, because this device doesn't support it.
RadeonRX (5): RadeonRX (5): Cannot print bridge configuration for PCI:0.12,0, because it is not a bridge device.
RadeonRX (2): Cannot enable blind prefetch for PCI:0.12,0, because this device doesn't support it.
Everything with the "INTERRUPT=No" ToolType set isn't really usable, it's just a workaround to be able to use the gfx card at all - but it will definitely cause problems with other hardware/drivers, especially on the Pegasos2 with everything, except for the VIA PATA controller, using the same interrupt #25. All other AmigaOS 4.x systems use separate interrupts for each PCI slot instead. The only usable variant is therefore your 5., not 6.
How to explain that with INTERRUPT=No he has higher framerates in the games ? Non-working interrupts by luck don't cause lock/hang, but only cause speed drop or some more heavy emulation involved ?
3. CMD Line as above. Current master plus OSDN patched master. INT's = Yes. Same as before can't get into workbench before the system hangs.
No. 3 works for me (OSDN repo + Interrupts on) however I do not see any difference between this and using the master repo with no patches. Is the version number different in the OSDN code? if it isnt, how can i tell that Im using the correct binary?
The way I did it was to create three duplicates of the qemu source folder so I could have all three versions compiled up at once and I just pointed my startup script to whichever one I wanted to test at the time.
I'm curious why we are seeing different results when we are on more or less the same setup. I guess the answer may lie in the motherboard and the IOMMU groupings and / or isolation.
On that note, I completely don't trust the weird behaviour and host crashes I was seeing earlier so I'm in the process of setting up from scratch on a more manual host setup (going to try nixOS but if I get stuck with that I'll go Arch).
How to explain that with INTERRUPT=No he has higher framerates in the games ? Non-working interrupts by luck don't cause lock/hang, but only cause speed drop or some more heavy emulation involved ?
I suspect that qemu (or vfio) is somehow failing to pass on every interrupt from the card. Missing every second vblank interrupt would halve the frame-rate. Only passing on 1 out of every 3 would cut the frame rate down by 1/3rd.
Another possibility would be stale data being in the interrupt's ring-buffer. The graphics card writes to a ring-buffer in RAM which is supposed to be cache inhibited. If that isn't emulated correctly, then it could end up reading stale data.
I've reviewed the RadeonRX interrupt code, and it's *NOT* set up to share an interrupt with any other device. That's something I need to fix...
@MartinW
The infinite loop problem when the driver goes through the PCI bridge chain is probably both a driver bug and an emulation mis-configuration. Fixing either of those two will let it work. Somehow the PCI bridge secondary bus numbers is sending the driver into a loop.
Please contact me via email (link), and we'll try to debug this.
kas1e wrote:@balaton [quote] I tried right now PCI network card from my x1000 in pegasos2, so proved to working drivers, working on os4, etc. The card marked as "RTL8139D".
I just go to prefs:internet, and replace for interface from my via-rhine.device on rtl8139.device. Reboot, reattach cable to PCI network : and everything works. I can run Odyssey, can visit sites, can ping google, etc.
Anyway, what is definitely true, is that RTL8139D PCI based card over rtl8139.device driver works for sure in real Pegasos2 , without hangs.
Just to be sure which version of the driver did you use for the test?
I only ask because I have come in contact with someone who wrote this driver for AmigaOs4.1 and may be able to help solve this problem under Qemu.
Just to be sure, which version of the driver did you use for the test?
For amigaos4 or for morphos ?
For os4: 53.6 (11/14/2016) For mos: 50.17 (07/24/2008)
But, Balaton's thread point out that issues happens on all OSes, so it's not something driver only related for one single OS, but some general issue shared across all OSs.
Quote:
I only ask because I have come in contact with someone who wrote this driver for AmigaOs4.1 and may be able to help solve this problem under Qemu.
I know who wrote those drivers: Costel 'Cyborg'. But then again, issue not with OS4 only, but with Morphos and Linux as well, mean it's not single driver's issue, but something in qemu
For os4: 53.6 (11/14/2016) For mos: 50.17 (07/24/2008)
But, Balaton's thread point out that issues happens on all OSes, so it's not something driver only related for one single OS, but some general issue shared across all OSs.
Ok, then we can rule out the possibility that the drivers are faulty and, as you said, something is missing in Qemu that leads to the unstable network.
I know you did the test on a real Pegasos 2 and network card. The driver version was for AmigaOs4.1, thanks for the information.
@DrProton I tried 7448 some time ago but it doesn't go beyond the boot it stays at the initial screen I also tried to keep the most recent updates of update 2. Without replacing the files. But it doesn't go beyond the boot screen.
CPU speed* instructions L1 cache L2 cache 601 60-120 MHz 3 per cycle 32 KB external to 1 MB 603 75-160 MHz 2 per cycle 2x8 KB 603e 100-300 MHz 2 per cycle 2x16 KB 604 100-180 MHz 4 per cycle 2x16 KB external to 1 MB 604e 166-233 MHz 6 per cycle 2x32 KB external to 1 MB 604ev 250-350 MHz 6 per cycle 2x32 KB external to 1 MB G3/750 200-450 MHz 3 per cycle 2x32 KB external to 1 MB 750CX 366-466 MHz 3 per cycle 2x32 KB 256 MB onboard 750CXe 400-700 MHz 3 per cycle 2x32 KB 256 MB onboard 750FX 600-900 MHz 3 per cycle 2x32 KB 512 MB onboard 750GX 733-1100 MHz 3 per cycle 2x32 KB 1024 MB onboard G4/7400 350-600 MHz 19 per cycle+ 2x32 KB supports 2 MB L2 cache 7410 466-533 MHz 20 per cycle+ 2x32 KB supports 1 MB L2 cache 7450 667-733 MHz 20 per cycle+ 2x32 KB 256 KB onboard, up to 2 MB L3 7455 600-1420 MHz 20 per cycle+ 2x32 KB 256 KB onboard, up to 2 MB L3 7447A 600-1500 MHz 20 per cycle+ 2x32 KB 512 KB onboard, no L3 cache 7457 867-1267 MHz 20 per cycle+ 2x32 KB 512 KB onboard, up to 4 MB L3 7457 used in some third-party Mac upgrades, never by Apple 7448 1.0-1.7 GHz 20 per cycle+ 2x32 KB 1024 KB onboard, no L3 cache
So there might be a problem with at least RX when used with shared interrupts that's the case on Pegasos2 but probably nobody tried this on real hardware yet. Or did @kas1e test that already? I've lost track. This might be fixable in the driver and until then one can disable the interrupts for the gfx card as a work around. Correct?
However does the same IRQ related hang happen with old Radeon or HD cards passed through? If those drivers don't have this problem and should work with shared interrupts then that still confirms we have some problems with interrupts in emulation. But I still don't understand what exactly and what causes it.
QEMU does not emulate caches so there should be no issues with cache coherency or such. Unless the driver tries to read data too early before the card finished writing it I don't think that could be a problem. It's more likely that interrupts are either missed or cleared by acknowledging other interrupts while other devices still should hold it raised. In current master everything is mapped to the 4 PCI interrupt lines and code in PCI emulation should track their state and keep everthing as it should but I did not write that code so don't know how it works however as it's generic PCI emulation used by all other machines it should not have serious bugs but maybe in pegasos2 we use it wrong. My original patches on OSDN try to track each interrupt source separately and combine them to the one interrupt so this should not have the above problems but maybe not all sources go through this so it could still miss some changes. The one patch work around was originally for MorphOS which expects level sensitive interrupts so that tries to convert level triggered interrupts to edge triggered by generating an edge when interrupts are still held high after a source is cleared. This may have a side effect of refreshing some state somewhere that helps on AmigaOS which supposed to handle edge sensitive interrupts. So either something works differently on real hardware than we expect or there's some problem emulating it correctly. (I'll continue in different post as this is getting too long.)
The network related problems under AmigaOS and MorphOS probably have different causes. Under AmigaOS it's likely related to the IRQ issue which is just triggered by network traffic generating lot of interrupts but the same could be caused by other interrupts sources but network seems to be one of the biggest sources of interrupts.
@MartinW or others testing patches: Can you please check if info irq in QEMU monitor shows any difference with these patches after about the same things done in guest like booting and playing the startup sound and maybe downloading a file or so. If it hangs before that then when it hangs. Go to QEMU monitor and do info irq a few times and see if the numbers are changing. Also note what graphics card is used and with or without irq. Maybe that could show something about the IRQ source but since everything is mapped to IRQ 9 it probably won't but at least we could see how much interrupts are generated in each case.
On MorphOS the network does not work from the start but it was found that disabling sound makes it work and starting sound after the network is up keeps it working so it's probably really an issue with sound emulation there that prevents network starting so maybe not related to interrupts. MorphOS uses the interrupt controllers differently so maybe it has different problems so testing with one OS may not give meaningful results for the other as these work differently.
@kas1e: Are you saying the network problem seen on AmigaOS (hang after lot of network traffic) can be reproduced with Linux guest as well? I've missed that part. Is there a way to easily reproduce this issue? Maybe it's easier to debug with Linux as we can get debug logs from the driver there or add more debug if needed.
So there might be a problem with at least RX when used with shared interrupts that's the case on Pegasos2 but probably nobody tried this on real hardware yet. Or did @kas1e test that already? I've lost track. This might be fixable in the driver and until then one can disable the interrupts for the gfx card as a work around. Correct?
Correct. The driver's IRQ hndler should return 1 if it handled the interrupt, and 0 otherwise. Currently, the RadeonRX (and RadeonHD v5) driver doesn't. Older RadeonHD drivers do.
Quote:
However does the same IRQ related hang happen with old Radeon or HD cards passed through? If those drivers don't have this problem and should work with shared interrupts then that still confirms we have some problems with interrupts in emulation. But I still don't understand what exactly and what causes it.
As said above, the RadeonHD driver does return 0 for interrupts it doesn't handle (pre v5). However, it has also never been tested on Pegasos II, because the OS never sees it (it's behind a PCI-to-PCIe bridge which either the firmware or OS bootloader doesn't handle properly).
MartinW's R9 270 is handled by the RadeonHD driver, so he can comment on how it works under qemu emulation.
Quote:
QEMU does not emulate caches so there should be no issues with cache coherency or such. Unless the driver tries to read data too early before the card finished writing it I don't think that could be a problem. It's more likely that interrupts are either missed or cleared by acknowledging other interrupts while other devices still should hold it raised...
No idea about the Pegasos II, but I remember having to set PCI interrupts to level triggered on my A1-XE.
I'm in the process of rebuilding my OS from the ground up with a manual VFIO setup as I'm really not happy after the hangs and reboots (in the host) yesterday that passthrough is 100% correct and the GPU is preoperly isolated. Last thing I want to do is be feeding bad information and wasting people's time!
Ideally I would like to completely blacklist the AMD driver from the host but I don't think I can as I don't think I have an NVidia card here any more and I really don't want to buy any more cards. I used to have a low end one but I think it went with a machine I sold. In theory with the device being bound to the VFIO driver it should be properly isolated and the host should never touch the card. If that was the case then I don't really understand why the host was hard crashing! I suppose there still has to be some interaction between the GPU and the host in order for it to function.
For now, whether it was a long-term solution or not (I guess not as joerg said), the R9 270 card with interrupts disabled would be good enough. But not when the fan is going full turbo all the time. RX doesn't yet work, but the fan management is there. I am unsure whether Hans trying to debug RX is a waste of his time or not? Maybe it helps real hardware? I'll leave that decision to him and mail him when my OS is back up.
News feed
OS4 Feedback forum
OS4Depot Feedback forum
Amigabounty forum
OpenAmiga forum
AmiCygnix forum
ABC shell forum
AmiKit forum
Cinnamon Writer forum
CodeBench forum
Digital Universe forum
Dopus 5 forum
E-UAE forum
Gnash forum
Ibrowse forum
JAmiga forum
Odyssey forum
OWB forum
Qt forum
SmartFileSystem forum
Timberwolf forum
TouchDevice forum
TuneNet forum
Unsatisfactory Software forum
).
