On emulation like UAE and QEmu it doesn't cache the contents of the disk, but the contents of the host system disk cache instead, which doesn't make any sense, makes it slower (only a little bit) and especially wastes a lot of RAM which could be used for something else.
QEMU is very configurable so you don't know what block backend an emulated disk is using. Image files are common but using real partitions or even remote disks is possible so you should not decide in guest file system driver if a cache is needed or not as it does not have enough info to know that. It will just see an emulated disk but cannot know what it corresponds to on the host. Maybe not even a config option is needed as there should already be a simple way to disable cache in guest if not needed, if nothing else then deleting diskcache.library if it has no prefs setting. But it may be more user friendly if it has prefs in envarc: not a firmware env variable which may be different on different machines.
Quote:
Especially on the Pegasos2 which doesn't support ExtMem and is therefore limited to only 2 GB RAM that's bad.
What is ExtMem and is that applicable on NG Amigas at all? All of these machines are limited to 2GB because of the memory map they have dividing the 32bit address space to memory and I/O so PCI cards are mapped above 2GB and adding more RAM would overlap those and the firmware would not detect it. The sam460ex has larger address space so it has devices above 4GB and could use up to 4GB RAM but firmware does not work with that so it does not boot with more than 2GB. Maybe it could be fixed but this seems to be a general limitation of all of these machines and even reaching 2GB is not easy on real hardware which mostly works with only 1GB. So maybe software written for these also expect that.
Not possible for the AmigaOne emulation as long it doesn't support NVRAM. Do the Sam460 and Pegasos2 emulations support NVRAM?
I think @LiveForIt maybe meant prefs stored in AmigaOS env: not firmware env variable. That should be possible on all machines but maybe not practical as file system is needed to access stored env so that's not available at initing file system but should be when initing diskcache.library so that could check it instead not SFS.
None of these machine implement NVRAM but adding it should not be difficult. Here's a patch serias showing how to save and load nvram settings to disk: https://patchew.org/QEMU/cover.1675297286.git.balaton@eik.bme.hu/ and before that a memory region needs to be created and added at the right address where the nvram needs to be. This is different for each machnie. For AmigaOne there seems to be a register enabling disabling it so that should enable/disable the memory region probably in hw/pci-host/articia.c and then do the same as above series to load and save contents when creating articia device or when nvram is written. There is a memory region type in QEMU which behaves as memory on read but io on write and calls a write method which can be used to save written data if there's a backing store. I'm not interested in writing that patch currently so if somebody wants to give it a try this is the rough design for it, the rest can be found in QEMU docs and sources.
But it may be more user friendly if it has prefs in envarc: not a firmware env variable which may be different on different machines.
🤣 ENVARC: is on the boot partition and can only be accessed after the boot partition was mounted. Just like for the config of disk drivers used for the boot partition (a1ide.device, peg2ide.device, etc., U-Boot config variables a1ide_xfer, peg2ide_xfer, etc.) configuring the file system used for the boot partition is only possible with firmware (U-Boot, CFE, ...) env variables, not with AmigaOS env variables.
Quote:
What is ExtMem and is that applicable on NG Amigas at all?
Most CPU supported by AmigaOS 4.x are 32 bit CPUs (only the X1000, X5000 and A1222 are 64 bit ones) supporting at most 4 GB of virtual memory. For compatibility to some old AmigaOS <= 3.x m68k software, a single, rarely used function in exec.library is limited to 2 GB, only 2 GB virtual memory are supported. Additionally IIRC the complete upper 2 GB of the virtual address space is used for PCI(e) address space, even if much less would have been enough. But all PPC CPUs supported by AmigaOS 4.x support more than 4 GB of physical memory. Because of the mentioned limit of an ancient exec function and the PCI(e) address space most and especially old software can't use more than 2 GB RAM, only new AmigaOS 4.x software using the "ExtMem" feature can access more than 2 GB of physical memory. It's more or less an ugly bank switching feature, but the only way to access more than 4 GB RAM on 32 bit CPUs. The Pegasos2 kernel doesn't support ExtMem, probably due to a firmware limit, but all other AmigaOS 4.x kernels do.
@joerg I get that configuring ide driver to access boot partition needs firmware env and configuring the file system may need that too. But we're talking about configuring diskcache. Is that active before boot partition is mounted? If not that could use AmigaOS env. Trying to decide about cache status in the file system driver seems like the wrong place as it's not file system but disk specific but I don't know how this works on AmigaOS so maybe there's no better place than the file system driver? I'd expect the simplest was would be to have a pref for diskcache to disable it either globally or per disk but this has nothing to do with the file system as each disk can have several different file systems on it and all of them may need to use same setting for cache so that setting cannot be in SFS config.
Quote:
The Pegasos2 kernel doesn't support ExtMem, probably due to a firmware limit, but all other AmigaOS 4.x kernels do.
OK how does that work on amigaone and sam460ex then? But if nothing can really use it then probably does not worth to try to implement it.
NVRAM needs to be implemented in all Amiga virtual boards. Two years ago I was looking for adding it on sam460 board. My impression was it was really not an hard task, but anyway I give up.. QEMU is a beast and I'm definitively not a professional coder.
But we're talking about configuring diskcache. Is that active before boot partition is mounted?
Yes, the first thing SFS does is (trying to) activate the diskcache, even checking if the partition is really a SFS formatted partitions and not something else (reading the first block of the partition with the partition identifier, "SFS\0" or "SFS\2" in the first 4 bytes of the block in case of SFS), is already done through the disk cache.
Quote:
OK how does that work on amigaone and sam460ex then? But if nothing can really use it then probably does not worth to try to implement it.
It's nothing AmigaOS specific, just using the segment registers or MMU on G2, G3 and G4 CPUs, or the TLB on 405 cores (440 and 460 CPUs). Several parts of AmigaOS 4.x use ExtMem already, probably the most noticeable one is the "RAM Disk:": On systems with ExtMem support it doesn't use the valuable first 2 GB of RAM for it, or only some tiny parts for administration data, but the actual data of the files on it is stored outside of the first 2 GB of the virtual address space. Only on access of the RAM Disk files parts of it are mapped into the virtual address space.
@Maijestro OK, I did not check. If diskcache.library is in Kickstart and loaded before the boot partition is mounted. Then it cannot use AmigaOS env only nvram. I guess that would work at least on pegasos2 even without nvram because that already has the text file in Kickstart to override values for peg2ide where this setting could also be added, but that's not very convenient especially when using BBoor because then after updating it on disk you also have to update the zip. This is a bit annoying but maybe only needs to be done once if using a global setting to disable disk cache altogether. (But for that isn't it enough to just comment out loading diskcaache.library from Kicklayout and then no setting is even needed?)
Adding logic to SFS to decide this for the user may not be a good idea because it does not work for other file systems and SFS cannot know what the host uses to back the emulated drive and if it caches that or not (like using physical harddisk not an image file). And it's a setting for diskcache.library not SFS.
Ideally the user should be able to set it per disk if that disk needs to be cached or not. Would it be possible for diskcache.library to load with Kickstart but when boot partition with prefs is mounted it checks the settings and disables cache for the disks it's told to ignore? That way all control is given to the user and no early config is needed and a normal prefs could configure diskcache.
But all this may be overengineering as we could just add to the install guide to comment out diskcache.library if you're using image files on the host because in that case caching again in the guest does not help if diskcache.library can be disabled that way.
But did somebody verify it's relly not needed? Getting data from the emulated IDE device may take some time so a guest side cache may still be useful so that should also be measured to make sure.
@joerg On ExtMem I was more interested on how this could work on the hardware side. Since these machines cannot take more than 2GB RAM as the firmware will not detect it how could ExtMem be supported on those machines? Where the extra RAM comes from and where would it be mapped and how does AmigaOS know about that if the firmware cannot use more than 2GB?
What is currently the easiest way to copy data from guest to host under Qemu?
It is important that it is a solution that can be used on all systems equally. Linux/Mac/Windows. I would like to update my installation guide, because there are always questions how to get the kickstart folder from the installed AmigaOs4.1 system.
For the installation Qemu Pegasos 2/Bboot Kickstart.zip must be created 2 times, once for the pure installation and once after update 1 and 2 so that also newer kernels/drivers are available.
Preferably I find the solution of @smarkusg easiest
After the successful installation you should look for a better way via sharing network folders..FTP/SMB and co. It serves only for the pure installation via BBoot/Kickstart.Zip
Today I worked intensively with "ufat" again and I noticed that when I copy files from the host via /tmp they are displayed without any problems under AmigaOs4.1, but I copy from the guest system AmigaOs4.1 to the host system MacOs The files are not shown to me in the /tmp folder. So it only seems to work in one direction.
Even when copying from the guest to the host system I get the following error message in the console:
Maybe make a different directory than "/tmp" for sharing. That's where data/files that can cause problems can sometimes land. I gave this as an example only.
Try somehow from your home directory where you know that another program won't put any data there when using QEMU, or if you want from "tmp" then subdirectory "/tmp/qemu". You have to mkdir create it for yourself.
Maybe make a different directory than "/tmp" for sharing. That's where data/files that can cause problems can sometimes land. I gave this as an example only.
Try somehow from your home directory where you know that another program won't put any data there when using QEMU, or if you want from "tmp" then subdirectory "/tmp/qemu". You have to mkdir create it for yourself.
he he ok that was the solution...I created the tmp folder in the user directory and it works perfectly from both host and guest sides. You can also name the shared folder whatever you want, I named it “Transfer”. Thank you.....
Communication between host and emulated raspberry pi is possible via ssh / sftp, just use port 5555, it gets forwarded to port 22 on the "pi" (example: ssh -p pi@localhost:5555), so you can exchange files easily. Just make sure that ssh is enabled via raspi-config.
So you want to use this emulation to set up some kind of "browserservice"? Maybe you take a look at Selenium Webdriver, which uses Chromium on the pi to do some web scraping. Did this some weeks ago, but to figure out how it's working caused me headaches. You need a debian chromium for arm64 package and the debian chromium-webdriver package for arm64, make sure both versions are the same (this is important, otherwise selenium won't work). Install them with apt and make sure both packages won't updated automatically via apt. I used selenium web driver with python 3.11 on my emulated pi, in headless mode, because qemu only allows 1G of memory.
With nvram.config which is in the Kickstart directory I tried to activate DMA as follows:
nvram.config is only supported in the classic Amiga version of nonvolatile.library.kmod, which seems to be used for the Pegasos2 as well, but not in the versions used on hardware with real NVRAM.
I'm not sure any more which option(s) display the transfer modes, but one of
With nvram.config which is in the Kickstart directory I tried to activate DMA as follows:
nvram.config is only supported in the classic Amiga version of nonvolatile.library.kmod, which seems to be used for the Pegasos2 as well, but not in the versions used on hardware with real NVRAM.
I'm not sure any more which option(s) display the transfer modes, but one of
Thanks! I would have assumed that it would work similarly to Pegasos 2 hardware by simply using the nvram.config. I have carried out all 3 types of commands, but I am still not sure which value shows me exactly which mode is currently being used.
Neuer Shell-Prozess 7
7.Workbench:> idetool -l a1ide.device
-------------------Unit 0 information------------------------
Flags : $00000185 - present, supports DMA, ATA, supports cache flush, interrupts used,
Xfer mode : best pio 12 (PIO 4, 16 MB/s) / best dma 69 (UDMA 5, 100 MB/s) / current 12 (PIO 4, 16 MB/s)
Total blocks : 62914560
Blocksize : 512
Power mode : 2 / idle (ready for operation)
IO1 / IO2 / BMCR @ : $1F0 / $3F6 / $0
spurious interrupts: 0
bounced misaligned cache line start DMA requests: 0
bounced misaligned cache line end DMA requests: 0
bounced non-contiguous buffer DMA requests: 0
*** Error opening device a1ide.device / unit 1
-------------------Unit 2 information------------------------
Flags : $00000185 - present, supports DMA, ATA, supports cache flush, interrupts used,
Xfer mode : best pio 12 (PIO 4, 16 MB/s) / best dma 69 (UDMA 5, 100 MB/s) / current 12 (PIO 4, 16 MB/s)
Total blocks : 62914560
Blocksize : 512
Power mode : 2 / idle (ready for operation)
IO1 / IO2 / BMCR @ : $170 / $376 / $8
spurious interrupts: 0
bounced misaligned cache line start DMA requests: 0
bounced misaligned cache line end DMA requests: 0
bounced non-contiguous buffer DMA requests: 0
*** Error opening device a1ide.device / unit 3
*** Error opening device a1ide.device / unit 4
*** Error opening device a1ide.device / unit 5
*** Error opening device a1ide.device / unit 6
*** Error opening device a1ide.device / unit 7
*** Error opening device a1ide.device / unit 8
*** Error opening device a1ide.device / unit 9
*** Error opening device a1ide.device / unit 10
*** Error opening device a1ide.device / unit 11
*** Error opening device a1ide.device / unit 12
*** Error opening device a1ide.device / unit 13
*** Error opening device a1ide.device / unit 14
*** Error opening device a1ide.device / unit 15
7.Workbench:> idetool -u a1ide.device 0
-------------------Unit 0 information------------------------
Flags : $00000185 - present, supports DMA, ATA, supports cache flush, interrupts used,
Xfer mode : best pio 12 (PIO 4, 16 MB/s) / best dma 69 (UDMA 5, 100 MB/s) / current 12 (PIO 4, 16 MB/s)
Total blocks : 62914560
Blocksize : 512
Power mode : 2 / idle (ready for operation)
IO1 / IO2 / BMCR @ : $1F0 / $3F6 / $0
spurious interrupts: 0
bounced misaligned cache line start DMA requests: 0
bounced misaligned cache line end DMA requests: 0
bounced non-contiguous buffer DMA requests: 0
7.Workbench:> idetool -d a1ide.device 0
-------------------ATA device information-------------------
Dev type : QEMU HARDDISK
Remov Dev : no
Remov Med : no (probably harddisk)
Revision : 2.5+
Serial # : QM00001
LBA : yes
Tot. Secs : 62914560
Tot. Size : 32212254720 bytes, 31457 mbytes, 31 gbytes
Main features :
IORDY : yes
SMART : enabled
Security mode : not enabled
Power mgmt : not supported
Packet cmds : not supported
Write cache : enabled
Look ahead : not enabled
Device reset : not supported
Write buffer cmd : not supported
Read buffer cmd : not supported
Nop cmd : supported
Download microcode : not supported
Read/write DMA queued cmd : not supported
CompactFlash cmd set : not supported
Advance power mgmt : not enabled
Acoustic mgmt : not supported
LBA 48 : supported
Minimum PIO cycle time : 120 ns
Minimum PIO c.t. (IORDY) : 120 ns
DMA : yes
Minimum reg. cycle time : 120 ns
mode 64 ( 16 MB/s) : yes
mode 65 ( 25 MB/s) : yes
mode 66 ( 33 MB/s) : yes
mode 67 ( 44 MB/s) : yes
mode 68 ( 66 MB/s) : yes
mode 69 (100 MB/s) : yes
mode 70 (133 MB/s) : no
PIO :
mode 8 ( 3 MB/s) : yes (always)
mode 9 ( 5 MB/s) : yes (always)
mode 10 ( 8 MB/s) : yes (always)
mode 11 ( 11 MB/s) : yes
mode 12 ( 16 MB/s) : yes
Xfer S/IRQ: 16 (Max. number of sectors that can be transferred per interrupt on READ/WRITE MULTIPLE commands)
Xfer S/MUL: 16 (Max. number of sectors currently set to transfer on READ/WRITE MULTIPLE commands)
7.Workbench:>
I assume that in the last section it is "DMA : yes" ?
News feed
OS4 Feedback forum
OS4Depot Feedback forum
Amigabounty forum
OpenAmiga forum
AmiCygnix forum
ABC shell forum
AmiKit forum
Cinnamon Writer forum
CodeBench forum
Digital Universe forum
Dopus 5 forum
E-UAE forum
Gnash forum
Ibrowse forum
JAmiga forum
Odyssey forum
OWB forum
Qt forum
SmartFileSystem forum
Timberwolf forum
TouchDevice forum
TuneNet forum
Unsatisfactory Software forum
