Login
Username:

Password:

Remember me



Lost Password?

Register now!

Sections

Who's Online
104 user(s) are online (63 user(s) are browsing Forums)

Members: 1
Guests: 103

Chris, more...

Headlines

 
  Register To Post  

« 1 ... 7 8 9 (10) 11 12 13 »
Re: NVMe device driver
Just can't stay away
Just can't stay away


See User information
@Georg
UNIT_VBLANK isn't different to UNIT_ECLOCK anymore since that's not even possible (except for the A1200 and A4000 versions of AmigaOS 4.x maybe): timer.device has to work before the gfx card drivers are started and get the monitor frequencies. All timer.device units are probably based on the same CPU timers.

@geennaam
Found the problem in SCSISpeed: It's limited to about 200 MB/s (2^32 bytes / 20 seconds) because of the ULONG count in SpeedTest().
Changing ULONG to uint64 count should fix it. Maybe updating the calculations and printing the MB/s is required as well.
Using "SCSISpeed MINTIME=1 ...", instead of the default 20 seconds, should work as well, without changing the sources.

Using ReadEClock() you get higher precision than "just" microseconds, however additional to converting it to (micro)seconds you have to check for overflows. GetUpTime() results can't overflow, ReadEClock() results do.
If you need even more precision than EClock you'd have to use performancemonitor.resource, which can measure CPU cycles.

Go to top
Re: NVMe device driver
Quite a regular
Quite a regular


See User information
@joerg

A nice. Thanks for the effort.


In the meantime I have created my own benchmark. It's rudimentary for now but I intend to expand it with a graphical printout like those nice PC tools.

Anyways, here's the result:

Samsung SSD Connected to the X5000 internal SATA port. At 32MByte and onwards it starts to divide the transfer in chunks according to the output on the debug terminal
SSDBenchmark V0.1

device
p50x0sata.device,

Read size
4096 bytes
Result
24 Mbyte/s

Read size
8192 bytes
Result
31 Mbyte/s

Read size
16384 bytes
Result
57 Mbyte/s

Read size
32768 bytes
Result
90 Mbyte/s

Read size
65536 bytes
Result
94 Mbyte/s

Read size
131072 bytes
Result
176 Mbyte/s

Read size
262144 bytes
Result
217 Mbyte/s

Read size
524288 bytes
Result
242 Mbyte/s

Read size
1048576 bytes
Result
249 Mbyte/s

Read size
2097152 bytes
Result
248 Mbyte/s

Read size
4194304 bytes
Result
247 Mbyte/s

Read size
8388608 bytes
Result
247 Mbyte/s

Read size
16777216 bytes
Result
247 Mbyte/s

Read size
33554432 bytes
Result
235 Mbyte/s

Read size
67108864 bytes
Result
112 Mbyte/s

Read size
134217728 bytes
Result
112 Mbyte/s


Samsung NVMe SSD:
Starting at 16kb I have to get the physical page addresses of the read buffer to create a " Physical Region Page" list and feed it to the NVMe controller. Getting the physical address of a page in supervisor mode creates a lot of overhead as can be seen from the transfer speed. That's why pipelining of commands is so important as can be seen for the larger transfers.
In the future I want to rewrite that part of the driver to use Scatter Gather lists. If the buffer isn't fragmented then this should reduce overhead severely.
SSDBenchmark V0.1

device
nvme.device,

Read size
4096 bytes
Result
111 Mbyte/s

Read size
8192 bytes
Result
217 Mbyte/s

Read size
16384 bytes
Result
60 Mbyte/s

Read size
32768 bytes
Result
109 Mbyte/s

Read size
65536 bytes
Result
195 Mbyte/s

Read size
131072 bytes
Result
198 Mbyte/s

Read size
262144 bytes
Result
484 Mbyte/s

Read size
524288 bytes
Result
658 Mbyte/s

Read size
1048576 bytes
Result
771 Mbyte/s

Read size
2097152 bytes
Result
845 Mbyte/s

Read size
4194304 bytes
Result
871 Mbyte/s

Read size
8388608 bytes
Result
878 Mbyte/s

Read size
16777216 bytes
Result
1108 Mbyte/s

Read size
33554432 bytes
Result
1284 Mbyte/s

Read size
67108864 bytes
Result
1389 Mbyte/s

Read size
134217728 bytes
Result
1479 Mbyte/s


Edited by geennaam on 2023/5/4 17:59:35
Go to top
Re: NVMe device driver
Just popping in
Just popping in


See User information
@geennaamQuote:
In the meantime I have created my own benchmark. It's rudimentary for now but I intend to expand it with a graphical printout like those nice PC tools.

Great! Are you going to publish it, or is it only for your own use?

Go to top
Re: NVMe device driver
Just can't stay away
Just can't stay away


See User information
@geennaam
Quote:
Getting the physical address of a page in supervisor mode creates a lot of overhead as can be seen from the transfer speed. That's why pipelining of commands is so important as can be seen for the larger transfers.
In the future I want to rewrite that part of the driver to use Scatter Gather lists. If the buffer isn't fragmented then this should reduce overhead severely.
For a start use something like
uint32 flags 0;
if (
writeflags DMAF_ReadFromRAM;
if (
== IExec->StartDMA(buffersizeflags))
{
   
// not fragmented
   
struct DMAEntry dmaentry;
   
IExec->GetDMAList(buffersizeflags, &dmaentry);
   
// physical address of the first page is dmaentry.PhysicalAddress
   // 2nd page dmaentry.PhysicalAddress + PageSize
   // etc.
} else {
   
// fragmented, use current code
}
IExec->EndDMA(buffersizeflags);


If you want to cheat in your benchmark tool allocate the memory with
buffer IExec->AllocVecTags(sizeAVT_TypeMEMF_SHAREDAVT_ContiguousTRUEAVT_AlignmentPageSizeTAG_DONE);

Go to top
Re: NVMe device driver
Quite a regular
Quite a regular


See User information
@Rolar

For now it is a tool to profile my code and see how I can ootimese the speed. But I do intent to release it as a standalone tool some day.

One of the things that I've learned is that a DoIO() with CMD_READ (32bit) is executed much faster is than NSCMD_TD_READ64 (64bit). No idea why this would make such a difference for OS4.

Go to top
Re: NVMe device driver
Just can't stay away
Just can't stay away


See User information
@geennaam
Quote:
One of the things that I've learned is that a DoIO() with CMD_READ (32bit) is executed much faster is than NSCMD_TD_READ64 (64bit). No idea why this would make such a difference for OS4.
There should be no difference at all in the exec.library functions (DoIO(), SendIO(), BeginIO(), WaitIO(), CheckIO(), AbortIO()), and only a very tiny one in applications and device drivers (uint32 lba = io_Offset / sector_size vs. uint64 lba = (((uint64)io_Actual << 32) | io_Offset) / sector_size).

Try if adding NONSD to the SetPatch arguments in S:Startup-Sequence makes a difference, if yes something is wrong in your driver not supporting some NSD parts correctly.


Edited by joerg on 2023/5/5 17:11:50
Go to top
Re: NVMe device driver
Quite a regular
Quite a regular


See User information
@joerg

In my driver there is no difference in how I handle both commands. It always handles the address as 64bit. I just additionally zero the upper address for CMD_READ to make sure that there's no garbage which could Lead to a wrong address. I also see no difference in execution time within my driver. This looks OS4 related.

Go to top
Re: NVMe device driver
Just can't stay away
Just can't stay away


See User information
@geennaam
Quote:
This looks OS4 related.
I've seen the sources and I can assure you that there is no difference at all between CMD_READ and NSCMD_TD_READ64, or any other command, in the exec.library functions.
However, the SetPatch/NSDPatch replacement functions of the exec IO functions are something completely different, and I never checked them. I would be extremely bad if it's still required to add a "ISNSD" config line for nvme.device in DEVS:NSDPatch.cfg since that would mean it still assumes that all device drivers are broken, but OTOH that wouldn't surprise me.

Go to top
Re: NVMe device driver
Quite a regular
Quite a regular


See User information
@joerg

Never mind, my stupid mistake. Forgot to handle the upper 32bit of the address. ( iotd_Req.io_Actual ) in my Benchmark tool.

Go to top
Re: NVMe device driver
Just popping in
Just popping in


See User information
@geennaam

I have currently quite peculiar problem with my NVMe drive... I have on it several EXT4 partitions which I use with Linux, and a large NTFS partition which can be accessed also from AmigaOS. If I hide the icon of that partition from WB screen using the "Workbench" prefs, booting goes without problems. But if I do not hide it, booting usually stops before WB is fully loaded (only a grey screen and the white menubar is shown) and the whole system freezes. Rebooting helps only occasionally.

Do you have any idea why this happens...?

Go to top
Re: NVMe device driver
Quite a regular
Quite a regular


See User information
@Rolar

No, I do not know how hiding icons affect amigaos. Maybe @Joerg knows what could be the issue.

This doesn't happen for amiga filesystems. So one thing I could think of is that the NTFS-3G/Filesysbox combo might not have been implemented with fixed storage in mind. And removable storage like USB drives might be loaded and mounted after all dependancies of NTFS-3G/Filesysbox are available.

Go to top
Re: NVMe device driver
Just popping in
Just popping in


See User information
@geennaamQuote:
No, I do not know how hiding icons affect amigaos. Maybe @Joerg knows what could be the issue.

When I tested this issue further, I noticed that hangups happen also when the icon is hidden, but with clearly lower frequency. The NTFS partition is of course still mounted. But if I disable 'nvme'device' totally, the booting problems seem to dissappear completely.

Quote:
This doesn't happen for amiga filesystems. So one thing I could think of is that the NTFS-3G/Filesysbox combo might not have been implemented with fixed storage in mind. And removable storage like USB drives might be loaded and mounted after all dependancies of NTFS-3G/Filesysbox are available.

Do you know whether it is somehow possible to disable the mounting of a certain partition during bootup?

Go to top
Re: NVMe device driver
Quite a regular
Quite a regular


See User information
@Rolar

Quote:
When I tested this issue further, I noticed that hangups happen also when the icon is hidden, but with clearly lower frequency. The NTFS partition is of course still mounted. But if I disable 'nvme'device' totally, the booting problems seem to dissappear completely.
Then it definitly seems timing related. What you can do is remove nvme.device from the kicklayout and mount the partitions with sys:System/Mounter instead. Make sure that you change the device in the tooltype to nvme.device.

Quote:
Do you know whether it is somehow possible to disable the mounting of a certain partition during bootup?


No idea. I don't mount myself but hand it over to the mounter.library. However, there might be a configuration file somewhere.

Go to top
Re: NVMe device driver
Just popping in
Just popping in


See User information
@geennaamQuote:
Then it definitly seems timing related. What you can do is remove nvme.device from the kicklayout and mount the partitions with sys:System/Mounter instead. Make sure that you change the device in the tooltype to nvme.device.

Mounter cannot use a GPT formated disk. The 'MassStorage' commodity can, but it is somewhat tedious to use as you have to use the commodity exchange to open it. Unless there is some way to use it from cli...? Then I coud make script for mounting nvme partitions.

Go to top
Re: NVMe device driver
Quite a regular
Quite a regular


See User information
@Rolar


Try unticking the automount flag for the partition.
Mount the partition by using a mountlist file .

Ran in the problem a long time ago.

https://forum.hyperion-entertainment.com/viewtopic.php?f=57&t=3780

Go to top
Re: NVMe device driver
Just popping in
Just popping in


See User information
@Spectre660Quote:
Try unticking the automount flag for the partition.
Mount the partition by using a mountlist file .

But how? Please remember this is not a RDB but a GPT disk - so you cannot use MediaToolbox for it!

Go to top
Re: NVMe device driver
Quite a regular
Quite a regular


See User information
@Rolar

You could try setting the flag from linux .

Prevent GPT partition automounting
The automounting can be disabled by setting the partition attribute 63 "do not automount" on the partition.

Go to top
Re: NVMe device driver
Just can't stay away
Just can't stay away


See User information
@Rolar

Quote:
The 'MassStorage' commodity can, but it is somewhat tedious to use as you have to use the commodity exchange to open it.
It can also be opened using the hotkey combination Ctrl Alt M, if that's any help.

Best regards,

Niels

Go to top
Re: NVMe device driver
Just can't stay away
Just can't stay away


See User information
@geennaam
Quote:
No, I do not know how hiding icons affect amigaos. Maybe @Joerg knows what could be the issue.
I don't know anything about mounter.library nor how FUSE/FileSysBox file systems work, but there used to be a problem with some file systems if the partitions are mounted without ACTIVATE, i.e. the file system isn't started immediately but only on first access of DHx:.
In this case the first access is very likely the one done by the Workbench trying to load the disk.info icon file.

Go to top
Re: NVMe device driver
Just popping in
Just popping in


See User information
@geennaam

I'm testing now an A-Data SX8200 PRO 512 GB nvme disk in my X5000. I cannot get the nvme.device to work wth it. If I have it activated in Kicklayout, booting halts always after kickstartmodules are loaded (just like when there is no nvme disk installed at all).

I tested the A-Data first totally unformatted, then GPT fromatted but without partitions, and finally with partitions (EXT4 and NTFS) on it, but no difference. From Linux it works normally, so the disk itself (and the Axagon PCIe x4 adapter) seems to be ok.

Can you see any reason for this, and possible a fix...? This ADATA model has a DDR4 cache on it. I tested earlier an A-Data XPG GAMMIX S11 Pro (without cache), and it seemed to work ok, as did the Kingston NV2 as I reported here earlier.

Go to top

  Register To Post
« 1 ... 7 8 9 (10) 11 12 13 »

 




Currently Active Users Viewing This Thread: 1 ( 0 members and 1 Anonymous Users )




Powered by XOOPS 2.0 © 2001-2023 The XOOPS Project