X1000 CFE Patches: Testers Welcome (Fear not: real-time, no reflashing) v.01

@sailor

I notice you are getting DRQ errors. I've only seen this when reading from optical disk with atapi device. Though it can affect any data transfer I've only noticed it when I had optical discs in my drive while booting CFE.

@nbache
yes, the commads was enclosed within quotes

@pvanni
@nbache

Quote:

There is a glitch when saving envars with the patch. kas1e already knows.
Save the envars without patch applied.

see post #33

@kas1e
@sailor

I was away a couple of days. I'll do the tests too and add them to the ones sailor already did.

@kas1e
Quote:
In case that helps I have this:
https://zero.eik.bme.hu/~balaton/qemu/qemu-X1000-test.patch.xz

To try it you need this command:
qemu-system-ppc64 -machine x1000 -serial stdio -bios cfe.bin -d unimp,guest_errors,invalid_mem,in_asm -trace enable="serial*"
But you would need to add a lot of PA6T emulation to make this boot anything or be able to test more than the CFE startup but maybe it's good as a start or for some simple testing/debugging but I think it won't get far enough without i2c and pci parts to even get a prompt in CFE.

I think with this patch CFE tries to detect the RAM but I don't see where it tries to access the i2c bus. I think it fails around 0xffe08c64 (easy to remember address due to ending in c64 ) where it attempts to access SMBus but gets something unexpected and never gets to poke the i2c registers. Maybe it could be found out with an attached debugger what it looks for but I haven't tried that.

In this patch I've also added a PA6T object where all the internal devices of the CPU/SoC could be implemented. The X1000 does not have a north bridge as it is embedded in the CPU. This is where it gets complex as the PA6T seems to have an internal PCIe bus where all the internal devices appear so this should be modelled accordingly in QEMU: create a PCI bus, implement PCIDevice subclasses for the internal devices and find out how to access their config registers in a memory mapped region (AFAIK this is a feature of PCIe so this should be implemented in QEMU already but I don't know how so need to look at how other machines with PCIe do it in QEMU to find out). For now I tried to just emulate the few registers it accesses but probably missed something. I think the SMBus device should have a BAR that contains the i2c registers but
I don't see it accessing it but stops when trying to read the vendor/device ids.

Due to the lot of missing devices to emulate in PA6T plus the south bridge (that is not too difficult but also some code to write comparable to the VT82xx models) it probably isn't worth emulating the whole machine but for CFE testing only the parts CFE accesses might be enough but could still be too complex. it's doable but the result would not be better than the already emulated machines so maybe it's only interesting for such testing as firmware development/debugging.

@kas1e

Quote:

Well there was talk for years about a CFE update that was prevented because the source was lost, missing, or source-napped and tied up in a dark vault far away being held for ransom. Now the source is free there are no excuses. The most important fix needed now is RX cards.

Quote:

Once all work is done it should be possible to compile a binary that can be flashed using CFE flash commands. But better than the last one requiring to enter cryptic numbers. A simple batch file would suffice to do the tricky bits.

Quote:

I imagine that would require to disassemble, modify source, and reassemble so that it all works as one unit. Similar to how ProTracker was made by reverse engineering SoundTracker and hacking it. Or a virus inserting code into a hunk file. Just the usual deconstruction and reconstruction methods.

The scenario I was thinking was having an RX card with a live patch loaded from SSD. But then something went wrong with SSD and CFE couldn't load or boot from it. In this case CFE cannot setup the card so the screen would be blank. The user would be clueless as to what is going on unless they have a serial terminal attached. With RX this is the case now since people using RX on X1000 need a terminal for boot menu unless they only have one Workbench or boot volume. Which seems unrealistic for those who use an RX card.

But, it's still like this in AmigaOS. You can have an incompatible card or even a glitch and it will show up with a blank screen. With no indication of what is going on or any error. I'd to see this explained to the casual user. How is it, for example, that UBoot loaded up and showed on screen but the Workbench won't load? I'm afraid excuses about graphic drivers not working or chipsets not supported won't cut it here. The card works in the firmware period so it can work. They have got a bit sloppy here and only catered to the advanced user. If there is an issue with screen drivers then there should be an ERROR on screen! At this point no graphic driver has reset Radeon card as it doesn't know how to. There should be a fallback so the driver can show a message to the user. Something more useful than a blinking cursor if you're lucky. Amiga rant over.

@balaton
Oh that awesome that some work already being done ! Of course there no needs for full emulator, but just for some minimal basics, so i can:

1). Run firmware so to have "prompt"
2). be able to type something in firmware minimal (like "help", "show xxxxx"
3). be able to run "reflash" from it.

So that will give me ability to test on qemu that it actually correct binary, loads up for x1000, and i can reflash from it if something going wrong. And work will be of this kind : i build patched CFE, run it over qemu, if it loads and i can type , so can reflash on real x1000. At least some basic protection and speed in up of tests :)

@all
2 days were busy with some crazy stuff: i take the X86Emulator from the Max's UBOOT which he use for SAMs machines (the one which support and HD and RX cards), and do this: load up it as patch to the memory, and do run POST init from this emulator (the one takes the most time), and then, patch the "vga init" code of CFE, so it skip internal CFE's "POST" and use my new one from new X86emulator : that was hardcore , because were in needs to create lots of proxies and all that "connection between" stuff.

So just hour ago were able to make it all works as expected. I.e.:



And have it working with our (uboot-max's) emulator.

What the point of it ? For first, it works for HD and RX out of box (for RX i anyway still need to worry about CFEs hardcoded 8bit, so to transfer from 16 to 8, or from 32 to 8). In the emulator itself i almost made no changes. For second : speed. I measure what i have now on HD card:

original CFE's emulator when i do "vga init -fb -keep" : 9.6 seconds till CFE prompt appear (so POST of emulator, set the mode, etc).

our emulator when i do:

boot -elf -noints 192.168.0.144:vga_init/vga_init.elf
vga init -fb -keep

7.2 seconds (+ take into account that i copy+paste second command, so those 0.2).

Yes, just 2.5s won, but now i have by hands full sources of emulator, i can try to optimize it somehow, maybe even wrote some simple JIT for the most time consuming instructions, so theoretically maybe have in end ~4s, which in compare with about ~10 seconds can be not that bad !

@Hypex

Sorry, i dont know what you talk about when says this:

Quote:

It's like you didn't read the first post and dind't get what i do there :) WE DON'T HAVE X1000's CFE SOURCE CODE. I do not know from where you got that idea that we did ? All we have is source code of ORIGINAL broadcoms's CFE , WITHOUT Pa-Semi/Hyperions's changes. If i have sources, why should i guess anything , i simple can know for sure how it all works.

Quote:

We HAVE NO SOURCE CODE OF x1000 CFE :) How can we compile it, if we have no source ?

All my work there done (as i explain in first post), on the DISASSMBLY and DECOMPILATION of CFE.bin by Ghidra, and, i use just as reference the original CFE's code from BroadCom, but this one OF COURSE not have anything about x1000 in. So it only used for REFERENCE.

That why i do memory patching, i install HOOKs , trampolines, just like viruses do , and extended/rewrite bits which need it to make fixes work. I should guess on the names of functions which is not in original's CFE but only in X1000 changes of CFE, because we have no source code of it !

Broadcom's CFE just have nothing about amigafs, or about any rendering to framebuffer, or anything else which is X1000 related, this is all walking in the dark of assembly.

I need to add, that decompiled .c files from cfe.bin (with of course many functions simple missing instead of raw data with ????, with missing functions names totally as CFE.bin sripped down of debug symbols) are alone 2 mb of size, and plain disassmbly 48 (!!!) MEGABYTES of size. To compile back anything from that is near to impossible, so all what i can do, is to write a PATCHER, which will patch current CFE.bin which then, we can reflash back. But we can't build a shit because we have no sources of x1000 cfe

Hope now its clear :)

@all

RadeonRX progress

After few days of tinkering in the dark with "what functions is mean to be in this damn disassembly" find out most of the vga-framebuffer functions, and at least can change size, make reroute to the 32bpp, etc, and start to see something with correct 800x600x32 bits, instead of 8 hardcoded bits:




What have now:




Mean text typing works, background image works. Issue keeps just with drawing menu box correctly + clearing of screen + some bits there and there. Then, speed this up (because its now 32bits instead of 8), but almost !

@kas1e

There is a way to enable 8-bit graphics on Radeon RX cards in U-boot / CFE

The 8-bit VESA modes have been removed from the RX cards BIOS, but the graphic card is still capable of supporting 8-bit displays (in fact, this is possible with AmigaOS 4.1)

Since the BIOS is loaded into RAM, in theory it is possible to modify the BIOS StandardVESA_Timing table and include the 8-bit modes.

This way, separate 8-bit and 32-bit text routines are no longer necessary, and you achieve faster write speeds on the screen

@kas1e
Quote:
I did this as a test to see what it would need but I don't intend to continue as it seems to need implementing a lot of undocumented PA6T devices and another south bridge emulation to be usable (these may reuse some code from existing emulations in QEMU but still quite some work) and the result would not be better than the existing machines so I'd rather improve those further instead of wasting time on yet another machine. So I'm sharing it if anybody wants to continue or can use it for something but don't expect me to give you an emulation.
Quote:
For a start CFE should be able to detect the memory. This either needs emulating the smbus/i2c bus to access SPD EEPROMs (there are existing emulation of these in QEMU they just need to be hooked up the the right registers) or maybe you could patch this out from CFE.bin and hardcode some memory size instead. Then it may try to init some more internal devices so those need to be emulated at least that much to pass those init and not crash on unexpected results read from not emulated registers.
Quote:
This needs replacing rom memory region with actual flash chip emulation. There are some flash chips emulated in QEMU, if you're lucky you could use those otherwise this is another thing to emulate.

Quote:
As you can see to fully test a firmware you'd need fairly complete emulation of the machine to catch problems. Without that QEMU may be more useful to debug issues with your patches or explore what the firmware does as you can attach a debugger to QEMU or use the monitor to explore memory or CPU state easier than on real machine.

@all
If any of you use GPIOLV10 hack for RadeonRX, can you plz check that : boot to cfe with serial console, and type "vga init -fb -keep", will you then simple crash if you will do something like "dir -fs=amigafs ide0.0:" ?

I just found that does not matter what card i use (hd or rx), and fully original unpatched CFE, when i boot with GPIOLV10 set, and do "vga init -fb -keep" , then any access to ide0.0: cause a 0x200 crash ! Be it simple "dir", or "boot" or whatever. And that exactly after "vga init -fb -keep", pure "vga init" not cause such a crash.

Can any of you confirm plz ?

Thanks!

@All
Ok, i made it ! VGA patch! But fistly, NOTE:

As we don't reflash CFE for now, but only patch in memory, that mean we should interrupt original video card detections in ANY CASE by GPIOLV10 jumper set. Because if not, video already will be initialized if it RadeonHD, and freezes if it RadeonRX, so while we not patch/reflash CFE.bin for real, you had to set jumper and add necessary line to startup as shown futher, but you will have working menu / amigaboot now in CFE, etc.

What this patch do:

-- Tell the CFE to not load original x86 emulator when "vga init" will be run.
-- Load up Max's UBOOT x86emulator supporting HD and RX cards with small optimisation for few more seconds
-- Made a proxy functions to connect CFE's code with new emulator
-- As RadeonRX have no more 8bit modes, but CFE are hardcoded to 800x600 8 bit modes, then for RadeonRX created replacements/patches for necessary functions so to convert from 8bpp to 32bpp (for RadeonHD this one not used, of course, and go old route)


So results from PowerON till working menu (not just boot image, but to menu) (all on 2000mhz), line:

set astate 4 -speed=2000; set pmu -astate=a4; boot -elf -noints -fs=amigafs ide0.0:vga_init.elf ; vga init; set console pcconsole0 ; menu

RadeonHD original CFE's emulator: 26s
RadeonHD + patch = 21s from power on.
RadeonRX + patch = 23s from power on.


Same, but together with UDMA fix in sata and loading till working WB from PowerON:

set astate 4 -speed=2000; set pmu -astate=a4; boot -elf -noints -fs=amigafs ide0.0:sata_fix.elf ; boot -elf -noints -fs=amigafs ide0.0:vga_init.elf ; vga init; set console pcconsole0 ; menu


RadeonHD original CFE: 48s
RadeonHD + sata/udma/new_x86emu: 32s
RadeonRX + sata/udma/new_x86emu: 34s


Keep in mind we didn't patch CFE for real, just in memory, which mean that 2000mhz is not set right at beginning, we had to load a patch to memory, etc. So realistically , with patched CFE.bin + reflash we easy can have 30 seconds till working WB or even a little bit less.

There is VGA patch if one want to play with:
https://kas1e.mikendezign.com/aos4/x1000/firmware/vga_init_v1.zip

Please report all the bugs you find.

Another idea is to try to add DMA for graphics output in CFE, so it all will be fast, and not slow motion when it come to scroll/fast redrawing/etc..

No luck here. With an hd card I can see the menu, with my rx550 only black screen.

@TearsOFMe
Are you sure you did all correct ? You seems not did all the steps.

After you run the patch, and it says VGA Init complete, you should manually run "vga init", and after that "set console pcconsole0 ; menu". It looks like you just run the patch, and then do nothing, or , simple run only "menu" , because you see menu on serial. I.e. your log show that "vga init" and "set console pcconsole0" commands wasn't run.

So try to do manually this:

When you in interrupted console (with this jumper set) type:



Once you see "vga init complete" , type:



Yes, you type "vga init" again. Vga patch only load up it's own emulator and do POST of bios, all the other "vga thing" done by CFE, so you have to run "vga init", and you will have something like:



Only after that you do "set console pcconsole0 ; menu" , so to switch from serial to monitor, and run menu from.

Try firstly all manually, command by command, then can cookie up it in one line.

ps. better to cookie up your outputs here with some code or quote tags, so to not have too big posts with outputs (to easy read)

I can't get the USB serial input to work, so i cannot type
the commands with the Rx card inserted.. I will try again. Thanks.

P.S.: Ok that works. I am so grateful for your work. Thanks.

My boot time now is around 24sec till menu then 23 sec till workbench. 10 seconds faster. With 3 ssd and one cdrom drive installed.

@TearsOfMe
Quote:

Uh , that cool ! Thanks you for tests :)

I for myself were pleased and feels like something magic happens when i see RadeonRX CFE output on X1000's screen. Like something which can't happens, but still is :) What RadeonRX card you have ? (my one 560 one)

@kas1e

Thankl for all this.

But to be frank, I do not quite understand what we must do.

Where do we post all these lines? In a serial terminal?

I think, when I have time and I have tested everything that is written in this thread, that I will have to write a step by step procedure because at the moment, I don't really know what we have to do.

Even with the GPIOLV10, we do not understand what we must do

@K-L
Quote:

Yes, that sounds hard until we made it all to be simple new flash binary which just reflash and that all..

So, for now what you do is :

1). install anywhere vga_init.elf (that patch), better to ide0.0 ffs boot partition where amigaboot.of placed (just this one prove to work).

2). when you reboot, and jumper bring you to serail terminal (if not and you have already some line for STARTUP, simple hit many times "ctrl+c" on serial terminal so to break to CFE promt), you simple type "boot -elf -noints -fs=amigafs ide0.0:vga_init.elf".

This one will load up our patch.

3). after it don , you simple run "vga init". It will run remain necessary CFE stuff.

4). After it done, you run "set console pcconsole0; menu" - that one to bring back to x1000 monitor and have menu on.

After it all work, you can made it as one single like for STARTUP so to runs automatically.

@TearsOFMe
Quote:

Hm 23 secs to WB too much .. Is it with latest sata_fix with UDMA fix in ? It should be about 15s max to full WB from amiboot : 1s max for loading all kickstart modules, and 13-14 for OS itself.

I think so. The s2 sata_fix.elf. kickstart loading is very fast maybe here 1-2secs.
Btw. I tryed with two cards (Rx,hd) but that ends in endless loop.

P.S.:I have yeston rx550. I will try tomorrow with a fresh os install.