@geennaam
Thanks again. A few more ideas that may or may not help:

- Maybe try to remove -device VGA option but then you 'd need a way to get input into the guest so you may either need to pass through an additional PCI USB controller too and connect mouse/keyboard for the guest there or you can try passing thorugh only a USB device plugged in the host with -device usb-host but then you'd need to configure the host to not drive those first which may be difficult. So you could try to use -device bochs-display,romfile="" instead of -device VGA,romfile="". The reason is that bochs-display does not have legacy VGA resources that may clash with the ATI card that also has these but those are not passed through now and may not be needed but I think that's what the x-vga=on option of vfio-pci controls so maybe you can try if that makes a difference. Even though the guest drivers probably don't need legacy VGA the card BIOS may need it and could get confused by having a QEMU VGA device there. Also some BIOSes may init cards that could cause problems so may chack if there are any relevant BIOS settings in the host.

- Instead of unbinding the driver before starting QEMU it may be better to blacklist the driver to prevent it from loading at all to make sure the device is in a state it should be after power up and no driver messed with it before the guest starts. For this you may need to edit /etc/modprobe.d or /etc/udev/rules.d on the host (or wherever your distro keeps these). I don't quite know what to put in there without searching for the answer so you should do the same to find out.

- I was told before that some cards with a reset issue may need some changes in the Linux kernel to handle them, such as in the Linux kernel source "drivers/pci/quirks.c" may need

and/or

and then rebuild and reload the new kernel but first check what's there for your kernel version and graphics card already.

I don't know if any of the above could help and it could well be we're missing something about handling DMA in QEMU that I don't yet understand but if you want to keep experimenting these are the directions I'd try.

@joerg
Quote:

I'd say instead you can try but don't expect it to work or don't expect it to be fixable. So sticking to what should work on real hardware at least at first has a higher chance of working or at least should be fixable if it doesn't work but I don't think we should limit ourselves to what's possible on real hardware if it can be made working. So I would not be as strict saying don't even try it. Just know what you can expect.

@geennaam
Thanks for testing and providing this info. This should not only help me to understand these but also Amiga people on this forum so I think it would be useful to keep this here than moving to private emails with Hans.

You get the screen output on the external monitor if the card ROM runs enough to init the card. The ok prompt is also there if you can't see it that's why you need to redirect to serial with /failsafe if the card ROM cannot run to get it back. But you could blindly type the boot command as well, this is what I tried to explain before.

I don't know MorphOS either, I only had experience with these OSes trying to get them run on QEMU but it's possbile that MorphOS driver handles the error and runs in some fallback mode or just does not use the same accel features. Only those would know who wrote these drivers. But I think there's some problem with setting up DMA as proven by the errors. I don't think this is normal and likely a problem with not emulating some parts of the chips related to this. On AGP there's GART and on PCI something similar I don't remember now but I would look at that direction and find out how these supposed to work then check if we emulate that correctly in QEMU. For this seeing the memory map and deivice settings might be helpful so I ask again to collect info mtree and info qtree ouput from QEMU monitor after booting these guests (or when they hang during boot with the HD card). You can get the monitor output in the shell window with -monitor stdio instead of -serial stdio so you can redirect or copy it from there more easily.

Later when we find out what registers should control DMA setup on pegasos2 (hope somebody here who already knows could tell that) we may enable trace output for those to see what the driver programs and check if it is handled correctly in QEMU which I think is missing to get DMA working correctly. QEMU usually only implements enough of the hardware that guests use but may be missing parts that nothing used so far.

@geennaam
The HD and Rx cards are not supposed to work on pegasos2 which does not have PCIe so it proabably fails because they have large memory BAR that is not handled correctly but Hans would know for sure what the SS = 3 means. The 9250 seems to be a better target first. If it shows SmartFirmware output then the BIOS has run and the card should be init'ed but you did not send much info so don't know what you're seeing. Was the serial output without errors with this card or did it still show some errors. From the brief report it seems BARs are mapped and basic communication and frame buffer works but probably there are some problems with DMA but I don't know how to debug that. You should really give more details on what you get from firmware before booting and also from QEMU monitor after booting to see how the card is set up. The AmigaOS debug logs are only useful to Hans as he's the only one who knows what those mean as nobody else has the sources so I can't help with that.

I know about a comment in qemu:
https://gitlab.com/qemu-project/qemu/- ... hw/vfio/pci-quirks.c#L243
that may be relevant but not sure if it would affect the AmigaOS driver.

You could try other guest OSes such as Linux as smarkusg suggested or MorphOS with the 9250 card as those have different drivers to see if they work or can reveal something. With Linux for example you can have a console on serial and explire the card from the OS even if you can't get picture so that can help debugging further. Also the Linux diver may log something more useful.

Also you never told what kind of PCIe-PCI bridge you have, it may also be dependent on that.

@derfs
If you make a mistake typing " /failsafe" io then it will stop working so make sure you type that correctly, space after first quote and space after second quote exactly as above with no mistakes. Once you typed that and pressed enter you can exit the guest window with Ctrl-Alt-G as show in the title bar and try pressing enter in the shell window where the errors are to see if an ok prompt appears there. If you typed the io command wrong then you don't have a second chance, restart guest and try again.

@geennaam
It would help if you could include info mtree and info qtree output from QEMU monitor and the card info as seen by the firmware, i.e. the ouput of following commands from the firmware ok prompt:

Probably the memory BAR is not recognised correctly which could happen because we might try to use an impossible config or the firmware is not prepared to handle it but it might be possible to patch the info in the firmware before booting.

smarkusg said you could test pass through with a Linux guest first, i.e. running a PPC Linux distro instead of AmigaOS in the emulated pegasos2 first as that may give you better debug info or if it works could confirm that the problem is not on the host/vfio/QEMU side but in the guest OS.

@joerg
Quote:
That command works but needs to define the variable first so you either need to boot cd bootloader_prepare first or just run the relevant command from that script which is:

before the above setenv.

@geennaam
We can't go any further from here without getting some more details about what is happening to try to find why it's not working. It would be helpful to test a config that's known to work on real hadware to avoid debugging problems that are not related to QEMU or VFIO so is this card knwon to work on a real PegasosII? Is there anybody with a real hardware and known working card that could be tested on QEMU?
Some ideas to get more details would be to enable some debug logging in AmigaOS4 if the Radeon driver has any. @Hans should know how to do that. Or trying to get memory map after the driver started to see if the BARs are mapped correctly. To get access to QEMU monitor you can either replace -serial stdio with -monitor stdio to redirect monitor to stdio instead of serial in which case serial window will be in the QEMU guest window from the view menu or CTRL+Akt+2 or 3. (There's also a -serial mon:stdio option which redirects both QEMU monitor snd serial to stdio and you can switch between the two with Ctrl-A c or see Ctrl-A h for help but this may be more confusing than redirecting just serial or monitor.) Once you get monitor on stdio you can try info mtree and info qtree commands to get info about the machine and see if the gfx card shows up in the guest where it should be but to make sense of that some knowledge on how this should work is needed so since only Hans knows what the driver does his help may be needed to debug this.

@Hans
Quote:
One thing it guarantees is that results won't be lost. If an open source project dnesn't succeed at least it can be taken over by others who can make something out of it an not lose the results while if a closed source project is failing then all their reuslts so far will be lost and since nobody can help them out they can struggle for a long time. The sam460ex emulation was started by sombody to test the Haiku Sam460EX port originally but was abandoned as it turned out to be more complex than a few weeks of hacking. But since it's open source I could pick it up and finish it but without that I may not even have started. Also much more people can use the results of an open source project than a closed one so a model where development is spnsored up front like in a crowd funding campaign but the results are then open sourced is better than doing something closed then hoping to sell enough to recover the costs. Doing crowd funded development with the results closed source but ninaries freely available is not ideal but still better than nothing. (Doing something closed with crowd funding and then selling it is the worst possible model IMO.)

Quote:
Fortunately we don't need any of those for a simple framebuffer driver so that should be possible to do as a first step. We'd just need somebody who either knows about this already or willing to read the docs and learn.

@geennaam
That likely means you're typing the boot command in the wrong window. When using -serial stdio AND not having a graphics card that the pegasos2 firmware recognises (such as sm502 which it does not know about) then the firmware console is redirected from screen/keyboard to /failsafe which is on serial by default so you see the ok prompt in the window where you started QEMU from and can type there.

But if there is a graphics card the firmware knows about it sets io to screen/keyboard even if it then fails to init the gfx card so you can't see output but also can't type in the serial window because the firmware is expecting output from the keyboard not serial. So you have to type " /failsafe" io command in the window where you'd see the output and keyboard is connected to, which is the one that normally says "Guest did not init display (yet)." at this point. If you don't get such window with only a passed through graphics card then add another card with -device VGA,romfile="" that the firmware will ignore but you'd get a guest window where you can blindly type the " /failsafe" io command to redirect input/output to serial or the boot command to continue booting.

Hope I could explain it enough now if not and somebody got it please try to explain it better.

@Maijestro

Yes QEMU supports all kinds of virtio devices already, only guest drivers would be needed to use them so this could be done by somebody who knows AmigaOS and needs no changes in QEMU.

@derfs
Quote:

That's not a lot of detail to understand what you tried and how it did not work. Did you get an ok prompt and could continue booting AmigaOS? Did the driver try to talk to the card? You might be able to get some detailed logs with enabling vfio traces with -trace enable="vfio*" to see if it does something at all. Passing through an older and simpler card first may have higher chance of working than starting with the newest most complex one.

Quote:

The linked datasheet also mentions that there are 2 PCIe buses, one 4 lane and one 1 lane. Fortunately the SATA is connected to the 1 lane and the 16x slot to the 4 lane PCIe bus so nothing is shared there. (Also we don't emulate the on board SATA so I would not care.) I think the first problem is that I did not fully implement the PCIe for sam460ex because we did not have use for it so far. I've started looking at cleaning up and enhancing that but as said there are other problems with sam460ex so trying to work with pegasos2 would be better.

Quote:

There seems to be still a lot to improve despite all that work so this may keep me busy for more years.

@Hans

Open sourcing the full Radeon driver may not be needed but publishing sample code for a simple frame buffer driver would definitely help. The docs on P96 API and driver writing docs were published recently by Individual Computers so if that's applicable for AOS4 too then these are available already but the sample code there is for 68k and supports a lot of stuff not needed for a simple driver. So a driver may be written based on that but for somebody without previous knowledge a simple example code might be more helpful to get started quickly.

Another part that would be useful to have open sourced is amigaboot.of which now needs the pegasos2.rom to run but if it was open source I could modify it to work with QEMU's minimal Virtual Open Firmware which can already boot MorphOS so if we had AmigaOS also work with that then we would not need the non-free firmware ROM any more. This could also help avoiding problems with the firmware as we could handle those in amigaboot.of which might even be useful for real machines later. But since I have no access to the source for amigaboot.of I can only reverse engineer and rewrite it from scratch to do that which would be simpler if this was released as open source.

I don't want to mention this further here to not derail this thread and keep it for discussing pegasos2 and sam460ex emulation in QEMU but in my view I could not write these emulations if QEMU was not open source so at least in some cases having open source is better. A large part of Apple's OSes are open source too and MorphOS also used AROS to reduce the work needed to write it so doing everthing closed is much more work than it should be needed otherwise and I'm not sure there's enough market left to support that work. But again let's not discuss this here to keep the thread on topic.

@geennaam
Quote:

At least that should work on real hardware according to https://wiki.amigaos.net/wiki/Pegasos_II
In the meantime you could try booting with your PCIe cards ignoring the errors from firmware as I wrote above (try " /failsafe" io to get the ok prompt back on serial where you can type the boot command or try typing the boot command blindly, see explanation in above posts) and see what happens.

@derfs

First of all what is gpu.rom, where did you get that and why is it needed? Can't it use the ROM on the card? It may need to be enabled from the host or QEMU run as root, I don't know but I'd just want to make sure the gpu.rom you're using is matching the card and should work at all.

The pegasos2 version at least tries to run the BIOS so it can at least see the card. Did you try to boot anyway and see what happens when the AmigaOS or MorphOS driver starts and if it can init the card itself? Since you have to type the boot command for pegasos2 you need to find out how to get input to the machine with a passed through card. Normally if you type in the window opened by QEMU that goes to the VM but I'm not sure what happens if you only have a passed through graphics card. Do you still have a QEMU window where you can type? If not you may try to add another graphics card such as -device VGA,romfile="" just to have a window where you can type the " /failsafe" io command or the boot command to get past the firmware and see what the OS driver can do with the passed through card, it would ignore the VGA device so that should not be a problem.

The sam460ex case does not seem to work at all, likely because we need more of the PCIe controller emulated. In the info qtree output the ROM is mapped at 0xa0000000 but if you check info mtree there's nothing there. Probably we need similar PCI windows for the PCIe bus that we have for PCI bus but I'm not sure which registers should control that. As I said I have no docs on 460EX so could only consult similar chips and those may work differently. The PCIe controller emulation is implemented at the very end of https://gitlab.com/qemu-project/qemu/- ... r/hw/ppc/ppc440_uc.c#L764 in case sombody wants to try to fix it just drop me an email if you work on it to avoid duplicated work.

Even if the sam460ex PCIe controller emulation would be fixed the CPU emulation on sam460ex is slower than on pegasos2. I've found the reason (related to software TLB emulation) but fixing it would need some more investigation and time so it's not likely it would be fixed in next QEMU release which means it may only be improved in 8.2 due in December so I think if it works with pegasos2 that may be better for now but you may have better luck with a PCI card than a PCIe card on pegasos2

@joerg @smarkusg

You are both right. There is a partial rv100 emulation option of ati-vga in QEMU but as almost all Radeon drivers use CCE it's not yet too useful. It only works with simple Linux framebuffer for now not with AmigaOS or MorphOS drivers so it would need more development to be usable that's why I did not advertise it.

@white
Quote:

See https://goteleport.com/blog/x11-forwarding/ for some explanation. I think you should replace hostfwd=tcp::2222-:22 with hostfwd=tcp::6010-:6000 in your command line and then use DISPLAY=localhost:10

@Hans
Quote:

I'm not sure the guest would see the bridge but I'm no expert on vfio. I think the bridge is handled by the host and it can pass the PCI card to the guest where it just appears as another PCI card so the guest does not need to know about the bridge but I haven't tried that. I've heard though that some people managed to do that and boot MacOS with a Rage 128 so I think it's possible.

Quote:

Depends how much we want to emulate it. Emulating the micro engine is really hard because it's not documented so I don't know how to do that. Emulating what the microcode does and parsing the command packets is only boring but not difficult but it was already mostly done in Xenia emulator so maybe it could just be ported to QEMU. It's still a lot of work but not much experties is needed just somebody who wants to spend time with it.

Quote:

That's what I said. I could try that if there was some sample template driver available somewhere. The P96 page has a cirrus driver as a sample but that's quite complex and written in 68k assembly so may not be the best reference. Some FPGA projects have simpler uaegfx drivers that may be used as an example but if there's some better reference to get started it might help. Driving the QEMU card is really simple because no initialisation or any such things are needed so I think a simple frame buffer "driver" would be like 5 lines the rest only being boilerplate code for AmigaOS needs. This is where I would start before looking at virtio.

@derfs

The errors you got with sam460ex are explained here:
https://bugs.launchpad.net/qemu/+bug/1869006
from that it seems it tries to map something at guest address 0xf0000000 and it may be due to a problem in some Linux kernel versions if I gat that correctly but I'm not sure what is happening here and if that's not a side effect of a diiferent problem.

Looking at the code that runs in sam460ex U-Boot at
https://gitlab.com/qemu-project/u-boot ... m460ex/init_radeon.c#L216
(that @m3x may know more about) it looks like it decides which BAR to map not based on card type but if it's a PCIe card. So maybe you should connect the card to a PCIe bus which may be possble by adding ,bus=pcie.1 to the -device vfio-pci options but since I had no docs on the PCIe on the 460EX this may not be correctly emulated but maybe worth a try. You can verify with info qtree in the QEMU monitor where the card is connected. Experimenting with ati-vga I've found only the PCI bus (pci.0) and pcie.1 are checked by the firmware).

Does anybody know if ATI X550 card is supported by AmigaOS? I may have a card around somewhere which I could test but I think it's not supported by either AmigaOS nor MorphOS although X300 and X1000 which is probably similar are supported.

@geennaam @derfs

For pass-through let's discuss the pegasos2 and sam460ex cases separately. For the pegasos2 the bios emulator may crash on some ROMs but regardless of this the machine may be booted and if the guest OS driver can init the card itself then it may work. I haven't tried pass through as I don't have a suitable card but I can reproduce the issue with ati-vga. As I wrote in the email Maijestro forwarded:



will result in similar errors but the machine works just the display output is not visible as the card is not init'ed. But you can type in the QEMU window and see more errors but you can also boot MorphOS and then it inits the card and see picture. You can get back the firmware ok prompt on serial when this happens by blindly typing exactly the following (note there must be a space between the first quote and /failsafe, that's Forth not a typo):



You have to type this in the QEMU window where the keyboard input would normally be and it won't be echoed but if successful then the ok prompt should appear where the errors are displayed and you can continue to type there to interact with the firmware. I'm not sure if you get a QEMU window with pass-thorugh but I think you should get some guest window, so type in that window not where the errors are. You only have one chance to type it without mistakes otherwise the io command breaks the input as well so if it did not work then restart the VM and try again.

You might also want to read https://osdn.net/projects/qmiga/wiki/DeveloperTips on the info qtree and info mtree QEMU monitor commands which may help debugging this and see where are the BARs of the card mapped.