The whole interrupt routing on pegasos2 is quote complex so I'm not sure I even understand it fully and it's quite possible we don't emulate that correctly somewhere. AFAIU the CPU interrupt input is connected to the MV64361 which has 2 independent PCI buses that each can have 4 IRQ pins but for the AGP port only 2 are connected. These PCI interrupts are connected to the GPP 12-15 pins of the MV64361 which are general purpose inputs that can be programmed to generate an interrupt. This should be emulated unless I've made a mistake writing that. However there is also the VT8231 south bridge that has a completely different IRQ routing as it's a PC south bridge containing usual PC hardware for ISA including 2 ISA PICs and peripherals routed there. The VT8231 also has 4 inputs for PCI interrupts and the interrupts lines for both buses area also connected here. These inputs can be routed to ISA interrupts within VT8231 with the other internal peripherals individually routable as well. The pegasos2.rom firmware routes everything to IRQ 9 though and guests don't change this so we also only emulate that. The VT8231 is also a PCI device itself and is connected to the pci.1 bus of the MV64361 so when an ISA interrupt is raised VT8231 raises its PCI interrupt EDIT: the VT8231 IRQ output is connected to GPP 31 of MV64361 instead, this is also correclty emulated, which is noticed by MV64361 and raises the CPU interrupt. That part probably works but then the interrupt should be also cleared at some point.
Clearing (aka ack'ing) interrupts is another mess. One can poke the PIC regs directly but there are also some virtual ack registers of MV64361 that can be read or written to ack an ISA interrupt within the VT8231 which I'm not sure is used but checking it now it's only implemented for pci.1 now and maybe it's not even correct in all cases. Since I don't know what AmigaOS and its drivers do I don't know what to check so some info on how interrupt handling is done and which way to ack interrupts is used by AmigaOS might help to debug and find where an interrupt could be missed. (If all the above works which I'm not sure it does yet then there's still a possibility that due to sharing interrupts there's some mixup happening when multiple sources change the same interrupt but I also don't know how to debug that. Problem with debugging these is one can't just add logs as there'll be millions of thees so it would be hard to find anything in there unless one knows what to look for in the first place. So to get further with this better understanding is needed what is happening and where could it fail then we can try to check that theory.
What I was trying to find out yesterday is what's missing from QEMU virtual open firmware for AmigaOS to boot. My boot loader would mostly be ready if that worked but I could not find out what it's missing. I can boot AmigaOS with it using the pegasos2.rom but with VOF AmigaOS does not start but crashes around the time it tries to access PCI devices. Not sure it it's still in the loader or in openfirmware.resource within the kernel but since it works with original firmware I think it's not something missing from the boot loader but something missing from the device tree under VOF. It does instatiate rtas and VOF has that as Linux also uses it and it works with that but AmigaOS does not seem to call the rtas yet but crashes between instantiating it and using it for PCI access. Does anybody know what it may do in that part and what it might need from the device tree at that point?
CI_CALL nextprop 3 1
CI_CALL child 1 1
CI_CALL peer 1 1
CI_CALL peer 1 1
CI_CALL getprop 4 1vof_getprop ph=0x4 "cpu" => len=-1 []
ph=0x4 "cpu" => len=4 [00000000]
CI_CALL finddevice 1 1 /cpus
vof_finddevice "/cpus" => ph=0x6
Invalid write at addr 0xFE000080, size 1, region '(null)', reason: rejected
Invalid write at addr 0x80, size 1, region '(null)', reason: rejected
invalid/unsupported opcode: 00 - 00 - 00 - 00 (00000000) 0182c350
Invalid write at addr 0xFFFFE1E0, size 4, region '(null)', reason: rejected
Invalid write at addr 0xFFFFE1DC, size 4, region '(null)', reason: rejected
Invalid write at addr 0xFFFFE1D8, size 4, region '(null)', reason: rejected
Invalid write at addr 0xFFFFE1E4, size 4, region '(null)', reason: rejected
Invalid write at addr 0xFFFFE1E8, size 4, region '(null)', reason: rejected
Both using my boot loader. It seems to get into a write loop or maybe pegasos2 firmware does something about setting up some MMU mappings on claim maybe? I'm stuck at this point at the moment.
Additionally if I break at the address giving me invalid 0 opcode at 0x0182c350 there seems to be code there at first but maybe something overwrites it so could be this is around when init-ing memory and uses some missing value that should come from the device tree but what?
Edited by balaton on 2023/7/16 15:02:25 Edited by balaton on 2023/7/16 15:25:58
@MartinW I never used vfio but read that problems with starting guests more than once can happen if cards are not reset properly and that's why there are quirks in Linux kernel sources for this. Maybe you should check host logs for clues and check what the kernel driver sources have about your card.
If this testing revealed bugs in the driver then debugging and fixing that would make it better for real hardware or in general so even if that would not fix the issues with emulation I think it's not wasted time.
Interrupts on pegasos2 are edge sensitive by default unless something changes that in PIC. MorphOS changes that using an old way that wasn't emulated but was fixed in QEMU 8.0 but I don't think other guests change it. One could check with info PIC and looking up the register docs for PIC to see what values mean.
I never used vfio but read that problems with starting guests more than once can happen if cards are not reset properly and that's why there are quirks in Linux kernel sources for this. Maybe you should check host logs for clues and check what the kernel driver sources have about your card.
It definitely felt like this. The reason I say that is because at least once the fans did not ome back up to full speed (remember this is the RX card where powermanagement works for the fan profile). I thought I had solved it at one point because the fans came back up when the OS ended but it didn't actually help. I will investigate. My "play time" is limited today, sadly.
Quote:
If this testing revealed bugs in the driver then debugging and fixing that would make it better for real hardware or in general so even if that would not fix the issues with emulation I think it's not wasted time.
As long as people are happy with expending the effort, then I'm happy. I just don't want to end up pushing people to spend their time doing things they'd rather not be doing. But the way I see it, there are only a finite amount of OS4 NG hardware boards left to sell. After that, it's emulation, or stagnation. But that's a different topic that has already been done.
Strangely I decided to re-read the thread as I didn't join from the start so I don't know why we're even trying GPU passthrough. Obviously if it worked then it would be the fastest emulated way until vfio drivers exist some day. I'm on page 9 and still haven't reached the point the passthrough discussion started yet
@MartinW There's no specific reason to try pass through I think just we were curious if it worked and how usable that would be. If there's any reason then that's the limited sm502 driver that made us look for alternatives and since reaching 3D with software (either emulating a 3D capable card or writing virtio drivers) may take a long time using pass through could be the shortest path to that now.
There's probably no reason to do any of this and it is just fun playing with these. Also nobody is pushing anobody to do what they don't want to do, I hope everybody just does what they find fun to do and is best at doing but if we put that together something useful may come out of it so we can have more fun with it in the future.
Maybe it doesn't have to emulate caches, but it definitely has to emulate the cache/memory access related WIMGE MMU/TLB/SR bits or it can't work correctly.
Quote:
If there's any reason then that's the limited sm502 driver that made us look for alternatives
It's not just the driver but the hardware that's incompatible to the AmigaOS graphics.library. AmigaOS only supports standard A8R8G8B8 32 bit modes (+ endian swapped versions), where alpha and RGB are combined in a single layer. The SM502 is different: It has a 32/16/8 bit RGB layer, and a separate alpha layer. Since something like that isn't supported by the AmigaOS graphics.library the AmigaOS SM501/2 driver can only support 15/16 bit modes, without alpha.
Edited by joerg on 2023/7/16 16:50:18 Edited by joerg on 2023/7/16 16:53:28
@joerg QEMU has softmmu to emulate MMU/TLB/SR, don't know what WIMGE is nor what all this has to do with caches.
There can be a reason for sm502 driver limitations but that does not change the end result that it's limited. I don't understand what you refer to but I think the separate alpha is only used for blending output with a video layer like genlock/color keying and in that mode sm501 may be limited to 16 bit. But for just display output it does support 32 bit RGBA mode (with the A part ignored) up to 1280x1024 so what prevents the sm502 driver to support that without compositing as it does in 16 bit mode?
@m3x The SM502 driver under AmigaOS4.1 is limited to 16-bit since its 32-bit mode AFAIK isn't support by graphics.library. The chip uses 4 bytes, 3 bytes for RGB, and the last byte isn't used at all. From my tests, AOS4.1 support 24-bit modes (3 bytes) or 32-bit modes (4 bytes with alpha) but not 32-bit modes without alpha. The result is that trasparency doesn't work.
This topic we already had under AmigaOs4.1 the "graphics.library" and probably also the driver would have to be adapted so that it works, since with the sm502 anyway no 3d acceleration is possible, this would be a waste of time, but at least one would have 32 bit windows.
The low resolution of 1280x720 doesn't bother me at the moment, host and guest use the same resolution because everything here takes place over a big 4k TV and I sit about 3 meters away from it, that's the best readable resolution for me
W = Write-Through Access (as opposed to copy-back access) I = Cache-Inhibited Access M = Memory Coherence G = Guarded Writes E = Endian swapping
Depending on the CPU those bits are either in the MMU table, in the TLB lookup caches, and/or the segment registers.
Especially memory coherence and guarded access is important for passed trough gfx (or any other passed trough PCI/PCIe) cards. At least for PCI config and I/O space it's required, not sure about MMIO. It's not required for emulated hardware like SM501/2, but it is for real hardware, like passed-through PCI(e) cards.
Quote:
But for just display output it (SM501/2) does support 32 bit RGBA mode (with the A part ignored)
Yes, and that's the problem: Any screen mode without alpha support, no matter if it's 24, 15/16 or 8 bit, can't support the most important AmigaOS gfx function IGraphics->CompositeTag(s|List)() and without support for that it's not much better than a dumb framebuffer driver.
@joerg OK so those would be something like generalised protection bits in the MMU. Since there's no cache in the emulated PPC most of these don't make sense so they likely do nothing with endian being the only useful one but since one can use ppc64le guests I think those are supported. (But does that even exist on G4 and before we're using here? I think those are big endian only anyway.) I think you meant these should be handled on the host not in emulation. But since vfio pass thorugh is in QEMU for a while and works for other machines including PPC64 machines emulated by QEMU I think problems with the basics in that is very unlikely. I'd expect problems within pegasos2 emulation (the parts I've written) which is just mv64361, vt8231 and the board code connecting these together. These are not fully emulated and there could be errors in there. In this case it's likely something with raising or acking interrrupts or tracking their state between multiple interrupts sources. It's hard to figure out though because this is scattered between different parts of code and I also don't know how AmigaOS usees this so what to check.
How does AmigaOS ack interrupts on pegasos2? Does it use the MV64361 regs for that or the ISA PIC regs? Maybe this is somewhere in the kernel and likely only who ported it to pegasos2 knows.
But for just display output it (SM501/2) does support 32 bit RGBA mode (with the A part ignored)
Yes, and that's the problem: Any screen mode without alpha support, no matter if it's 24, 15/16 or 8 bit, can't support the most important AmigaOS gfx function IGraphics->CompositeTag(s|List)() and without support for that it's not much better than a dumb framebuffer driver.
What does alpha support here mean? The SM501 does have a 2D engine which can do Windows ROPs including compositing so maybe the driver should use that but it doesn't and the defaults in graphics.library can't do software rendering in 32 bit mode only in packed 24 bit mode? I don't get all this but I don't know anthing about graphics.library especially the parts related to RTG but maybe this is helpful for somebody who understands all this. I still believe the sm502 driver is limited as even as a simple framebuffer supporting 32 bit mode could help. If that's possible for @m3x to do easily then it may worth trying unless it's impossible for some reason which I'm still not convinced about but it may be too much work to do.
I can't really update much today as I can't build QEMU from source at the moment, so take the following info with a pinch of salt as they say.
Vanilla QEMU v8.0.0, not from master and no patches. RX580 on pci.0 addr1. Without interrupts. It is incredibly, incredibly slow doing anything drawing related. Basically not usable, but it doesn't crash as such. With interrupts the system crashes before it even reaches the workbench, but the startup sound does play. Note that when this happens all IRQs except 9 are still active.
Just to summarise on the RX front. There is no usable scenario for me at the moment. Bus 0 is as described here, bus 1 needs debugging with Hans. I would prefer to get back to an OS I can build Qemu again from source. Sorry, I wasn't expecting that to be problematic.
[edit] Just noticed this on the debug output for RX580. Might have something to do with the slow speed. May or may not be related to fudges I have made in the firmware:
RadeonRX (0): System doesn't have full memory coherence. Compensating...
RadeonRX (0): Platform doesn't have full memory coherence, Disabling GART.
RadeonRX (0): No VRAM object for PCIE GART.
Radeon R9 270X may or may not have a usable solution right now. I think it is workable from a "does it function" point of view as long as you disable interrupts. This as Joerg pointed out is likely no good longer term. I'm a little confused on replies from Hans. I have a suspicion from the explanation of how the interrupt handlers should return 1 or 0, the v5 drivers might not work and the earlier ones would.
From a practical point of view, the v5 driver is needed to control the fans on the R9 270X regardless of the interrupt situation and I've reached my spending limit, at least for this month so I'm not prepared to buy the newer driver to find out whether the interrupt code works or not. Maybe next month.
The very old HD5450 card is probably the most usable for me to start some of the tasks I wanted to do on AOS, but I can only make a start as there's no 3D (honestly can't remember about 2D acceleration). The only advantage that really gives me over my 13" Mac laptop is bigger screen and a working network. They're pretty big advantages mind you
@MartinW I don't understand something. You changed the Ubuntu linux distribution just because of the linux kernel ? The problem is on the side of the module that you initialize at boot ?
I ask because you lost some time and I don't know if it was necessary if that was the reason.
When I started this I installed Ubuntu onto a spare SSD, ran a script from someone else and off I went.
When I started to see strange behaviour regarding BIOS reset on the passthrough GFX I wanted to be sure of the setup. Foolishly I also quite liked the idea of switching to NixOS and it's promise of an OS built entirely from a config file that later I could just backup and take to a different machine, etc., etc. I didn't think for a second that building Qemu would be problematic. It looks like it needs allsorts of workarounds and adoption of their way of doing things and I simply don't have time for learning that. Maybe another time.
Anyway, I'm happy that I now understand what is going on with the passed through GPU and that it's not the config. I still want a more stripped down OS without bloat though. I'll install something else in a moment. I've been using Linux since Slackware came out in the 90's so it doesn't take me long to backup and switch. I'll probably be back up and building Qemu by the end of the day.
@MartinW Aha ..ok "a man learns from his mistakes ... and a wise man from other people's mistakes". Good luck then, maybe your lesson will be useful to someone
Still trying to find out how to boot AmigaOS without pegasos2.rom I've made some progress. I've figured out how to pass os4_commandline in the boot loader so I could enable debug logs which got me very detailed logging duting the start. It's so detailed that it crashes now even with board firmware in some debug code so I've learned setting debuglevel too high can make it not even boot. But at least up that point I see no other clues on why it might ctash later so next I'll try to find a lower level where it still boots and hope that would still give me enough details to find the missing piece with VOF. That's for when I'll have some more time but at least there's a way to try further at the moment.
MartinW wrote: Radeon R9 270X may or may not have a usable solution right now. I think it is workable from a "does it function" point of view as long as you disable interrupts. This as Joerg pointed out is likely no good longer term. I'm a little confused on replies from Hans. I have a suspicion from the explanation of how the interrupt handlers should return 1 or 0, the v5 drivers might not work and the earlier ones would.
Radeon HD v5 drivers are not available for Pegasos2 machines, so we are limited to (max) version 3 drivers.
Quote:
From a practical point of view, the v5 driver is needed to control the fans on the R9 270X regardless of the interrupt situation and I've reached my spending limit, at least for this month so I'm not prepared to buy the newer driver to find out whether the interrupt code works or not. Maybe next month.
This is an issue that others have had with Radeon cards in real machines. I have never had this issue with a R270, R270x or RX580 in a X5000, or the R270 being used for passthrough.
Radeon HD v5 drivers are not available for Pegasos2 machines, so we are limited to (max) version 3 drivers.
Damn, yes, of course we are! Sorry, I'm forgetting myself now.
As for the fan, maybe I was just "unlucky" that the card I found for cheap on fleabay was the 270X. I mean, I was lucky that it came up cheap when all the others were at least 2x the price or more, but maybe a 270 might not have had the fan issue. Or it could even be make and model specific I guess.
News feed
OS4 Feedback forum
OS4Depot Feedback forum
Amigabounty forum
OpenAmiga forum
AmiCygnix forum
ABC shell forum
AmiKit forum
Cinnamon Writer forum
CodeBench forum
Digital Universe forum
Dopus 5 forum
E-UAE forum
Gnash forum
Ibrowse forum
JAmiga forum
Odyssey forum
OWB forum
Qt forum
SmartFileSystem forum
Timberwolf forum
TouchDevice forum
TuneNet forum
Unsatisfactory Software forum
