@Maijestro

Sadly, there is some really old legacy code that uses the sign bit (bit 31) of the return address to indicate failure. Until that component is rewritten (and all the old 68K programs that depend on it are rewritten or replaced), we can't use any addresses beyond (2 GB - 1).

@eliyahu
Quote:

Surely you jest! That is the best technical writing I have seen from any Amiga outlet, and for some years, I headed a documentation group for Defence projects.

Well done, that man!

Very curious.I've never seen anything like it on any of my machines, using any of my varied assortment of video cards.

The fact that several people have seen it suggests that there is a common cause, and I am pretty sure it is a software cause. I would first revert to the standard GUI theme, as many unexplained GUI effects over the years have occurred with custom themes.

I would next check the versions of graphics.library and diskfont.library, but that is less likely to cause such a problem.

Does it only appear in the window title, or is it visible on other text in the window?

@VooDoo

The X5000 and A1222 clock chips behave slightly differently from the chips used in earlier machines and the battclock.resource had to be upgraded for them. Not all machines were affected, it worked OK about 50% of the time.

As salass00 said, there was a new version of battclock.resource released to betatesters last year to fix this problem. It has not been released to the public yet.

I have spent the last week with an experimental version of NGFS that can bypass the cache when reading large transfers. Depending on the criterion for reading directly rather than through the cache, I can get speed increases for different transfer sizes. Some speed increases (for particular Read transfer sizes) can be as much as two to four times.

I have not attempted to allow Writes to bypass the cache because of the extra work involved to avoid clashes.

Overall speeds of (say) compiling the file system are not improved to any extent. Overall speeds of my test suite are not improved either. I have not tried profiling movie playback.

The show-stopper showed up today in the way of intermittent Read errors in one of my tests. If a caller writes a small update to an existing file, that change will be cached and will remain in the cache (not on disk) until the cache contents are flushed to disk. If, before the flush, a large Read comes in, that Read will get the old contents of the file directly from the disk, not seeing the update.

Since the Fast read shows speed improvements only for a narrow range of benchmark tests, and introduces problems that can only be fixed by adding code for particular conditions, the result is not worth the effort. I have abandoned the experimental version. Maybe one day a flash of inspiration will hit me, but in the meanwhile, I am happy to leave the FS as it is.

So I modified NGFS' ReadData() function so that for a Read request size larger than MAX_CACHE_READ, it bypasses the cache and reads the device directly into the caller's buffer. I haven't made any changes to Write yet.

Result is surprising: read speeds fall by a factor of 4 or 5. I then tried breaking up the long Read into several shorter Reads, but the overall speed doesn't change much with different sub-read sizes.

I think what is happening is this:

In the current version, everything goes through the cache. So the first read is slow, then all later reads are much faster, leading to an average that is pretty good. But when you ignore the cache and read directly from the disk each time, it's going to be much slower than reading from the memory-resident cache.

The code in DiskSpeed measures the overall time to Read() and Seek() to the beginning again (repeated many times). The actual times of the first disk Read() and the subsequent cache Reads are all averaged, so the difference between them is not visible. Bypass the cache and you see only slow transfers.

In the case of Writes, they all write into the memory-resident cache, which is written to disk some time later, so short Write() operations appear fast. They only slow down when the Write() length exceeds the cache size. A 1 MiB test size operates at full Write speed, although the reported speed is going to be slower than the maximum because of the included Seek() times.

I will add some longer test transfers to DiskSpeed and see what happens.

@msteed

To print left/right arrows in text, try using "ampersand lt ;" or "ampersand gt ;".

I'll leave it to you to change the ampersand to the actual character (and leave out the deliberate blanks).

@joerg

Thanks for that discussion. I think I tried (years ago) bypassing the cache for large transfers but it did not benefit the overall speed of the test suite, so I removed the extra code (don't like special cases). Of course, the test suite does not use a lot of huge transfer sizes such as we are testing with Geennaam's driver.

I'll try re-enabling the bypass-cache code and see if it improves DiskSpeed's performance.

I keep asking myself: "Why are we striving for maximum benchmark performance if it won't make much difference to real-world operation? What sort of application will benefit from an increase of transfer speed for buffer sizes > 1 MiB?"

I can't help thinking that this whole investigation is a solution looking for a problem.

@joerg

The transfer size limit is set by the size of the disk cache, the read-ahead cache and the number of available "buffers". Since NGFS has a write-through cache, all Reads and Writes go through the cache. Also, since it is a journalling file system, all Writes to disk (of meta data) take three Write operations, not just one.

The cache "buffers" are permanently allocated from the system and controlled by internal allocation code. Allocating and de-allocating cache buffers from the Exec imparts a heavy speed penalty. For a partition of 100 GB+, 4096-byte blocks are used, which requires 16 MB of cache for each such partition. I have 23 such partitions on my X-5000, so the cache is no bigger than necessary.

Many years ago, when I spent a lot of time optimising performance, I played with cluster sizes, number of cache buffers, etc. The FS was optimised (at the time) for overall speed *of my test suite*, not for the speed of individual transfers.

I have a test suite that runs all sorts of different tests and takes about 12 minutes to complete. The optimisation work was performed on a Sam 460 with a mechanical hard drive (the mid-range machine at the time). The 32-block cluster that limits read/write transfer sizes gave the best *overall* performance at the time.

Now that I have Geennaam's driver working, I can revisit the speed optimisations and check to see if there is anything to be gained by changing the settings. I doubt that any great increase can be achieved.

PS. Naturally, the test results I published were taken using the current version of NGFS. It would be unfair to publish the results of tests performed on other file systems. The partition size in this case was about 120 GiB.

I have updated DiskSpeed (not SCSISpeed) with the changes suggested by Joerg (to fix the counter overflow problem) and added a 1 MB buffer setting for test.

I have submitted DiskSpeed V4.5 to OS4Depot for upload, should be available soon.

Meanwhile, here are results of Geennaam's driver with a 512 GB Kingston "device" on my X5000-20:

>DiskSpeed drive NVMeDrive: all
DiskSpeed 4.5, OS4 version
Copyright © 1989-92 MKSoft Development
Copyright © 2003-04 Daniel J. Andrea II & Stéphane Guillard
Modified June 2023 by A. W. Wyatt for VP DOS API

------------------------------------------------------------
CPU: X5000-20 AmigaOS Version: 54.56 Normal Video DMA
Device: NVMeDrive: Buffers: <information unavailable>

Testing directory manipulation speed.
File Create: 5610 files/sec
File Open: 56.79 kfiles/sec
Directory Scan: 66.18 kfiles/sec
File Delete: 7578 files/sec

Seek/Read: 423.75 kseeks/sec

Testing with a 512 byte, LONG-aligned buffer.
Create file: 30.60 MiB/sec
Write to file: 42.26 MiB/sec
Read from file: 190.60 MiB/sec

Testing with a 4096 byte, LONG-aligned buffer.
Create file: 74.44 MiB/sec
Write to file: 288.38 MiB/sec
Read from file: 709.14 MiB/sec

Testing with a 32768 byte, LONG-aligned buffer.
Create file: 95.62 MiB/sec
Write to file: 1.15 GiB/sec
Read from file: 1.05 GiB/sec

Testing with a 262144 byte, LONG-aligned buffer.
Create file: 98.28 MiB/sec
Write to file: 1.58 GiB/sec
Read from file: 1.08 GiB/sec

Testing with a 1048576 byte, LONG-aligned buffer.
Create file: 95.12 MiB/sec
Write to file: 932.00 MiB/sec
Read from file: 986.50 MiB/sec

@geennaam

Yes, those #defines are in the headers but not implemented in the code, I have been told.

@geennaam

I have it on the best authority that the kernel does not, and never has, supported MSI interrupts. Nothing has changed in that regard with newer kernels.

@Georg

OK, it was not forbidden by law or documentation in those days, but it is dangerous and certain to cause a crash some time.

The only safe way to get the Next node AND remove the old one from the list is to:
(1) Get the next node first;
(2) Remove the old node.

Of course, you should have unique ownership of the list when you do that, to prevent other tasks interrupting and changing the list while you are trying to do the same.

We are trying to reduce the amount of internal structure that is exposed to the user. What was tolerated in the days of 68k assembler is often deprecated now.

@sailor

Don't use "debuglevel = 7" unless you want every boring detail printed out and have plenty of time to capture it all.

"Debuglevel=1" is quite adequate for almost all purposes. It's what I use for normal use and even debugging of device drivers and file systems.

@Rolar

ImageFX has always failed this way on startup with a debug kernel. It just shows what sloppy code we got away with back in those days. In this case, accessing the Prev/Next pointers in a node that has just been removed from a list. Another frequent coding error is to free some memory, then access something within it in the hope that it hasn't yet been destroyed.

The debug kernel shows up these and other glaring errors, at the expense of a slight speed decrease. Any programmer worth their salt will only ever run with the debug kernel, as the "release" version is likely to crash the whole system in the same circumstances.

Fortunately, the Grim reaper allows the user to "Ignore DSI Errors" and continue from a crash. ImageFX runs OK after that.

Seems the lack of interest on aw.net has caused this spammer to come over here to try his luck!

@geennaam

I tried a 500 GB NVMe card in an M.2 to PCI-e adaptor. As I indicated to you in private email last week, the X5000/20 crashes at boot with the driver in Kicklayout. It does not matter in my machine whether the adaptor card is plugged in or not, it still crashes.

I tried the adaptor card in the x16 slot beside the video slot. I have not tried it in any of the X1 card slots. I also have not tried the card in any other machine.

I have every NG machine here, so I can check the driver in any platform and provide crash logs or other data, if you wish.

It is possible that my up-to-date platforms behave differently from the older installation you are using for development. Another reason to investigate.

@Antique

Was that on your X5000?

If so, which slot?

The driver crashes my X5000, with or without the adaptor.

@mr2

Are you sure that there is some unused area on your SSD? The device firmware likes to move data around to minimise wear, and it can't do that easily (read: fast) if the device is full or almost full.

Best to leave at least 10% of the device empty.

@sailor

Yes, NGFS has so far only been released to X5000 owners.