Decreasing compatiblitiy of OS4 a good thing?

I
As a question: I am not really sure about the issue of moving forward.
If OS4 becomes "differently compatible" in the future, why is it so problematic for some people?

The current version of OS4 won't suddenly stop working! If any incompatibility arises a second partition on the internal storage with AmigaOS 4.1 Final Edition Update 3 will "fix it".

Then there are all the "Misters" and "Misses" and Vampires and #?A(6|12)00#? out there, that can run AmigaOS 3.3. So what is the problem?
Our current "NG" hardware should be NG.
No need to cripple it just for the sake of wanting to use 68k software.

Maybe I am alone here, but I would like to use my X1000 and X5000 and the coming Mirari at full potential. With all the cores available on the CPU-die.
And I would really like to have a system that does not "GrimReap", partially freeze and then leaves me with no possibility to save my work because the mouse and keyboard stopped working.

Stability has come a good way, but we are not there yet. Especially when running 68k software.

I am not a developer, but a fairly modern Webbrowser seems really improbable to me when I read about all the hurdles on the way. The endianness, Unicode, memory protection, and I think it is a matter of time until we are at a wall again with 32bit vs 64bit.
About the memory there is this Extended Memory Object, yes, but it somehow feels as a clumsy patch to me.

I read so often that "we" would like to watch movies and watch YouTube (or whatever streaming platform) but how do we achieve this with one core on a (here some cheers for you all PPC-is-dead aficionados) 10-years-old-dead-PPC-Processor?
Using 6 years old Graphicscards?
They are starting to post 8k video on YouTube, 4k becoming quasi-standard.
And besides videos, there really is the need of raw-power to decode some websites today!

Either you all folks have hope and believe new Amiga-hardware will be released with a super duper mega processor with Mega-giga-performance on _one_ single one of its x-cores, or we should hope that the OS will make full use of our still powerful "dead PPC "double or quad-core processors.

@kas1e

Quote:

Yes, I remember, thank for the clarification.

Quote:

I indeed would like 64bit, unicode, proper multicore support and whatever magic to make the system stable, with the mouse and keyboard (so the USB-stack) working. More then a couple of times the keyboard/mouse stop working while I can see the clock ticking, but no way to do anything.
Or the GrimReaper showing up, and then it's better to not click it, but close everything and reboot. Because clicking on "kill app" will brings the system down anyway.

I concur with you with sandboxing 68k and make it easier for developers to develop, debug and port software.

Also like many here, I don't want a Linux "clone". I think the NG-kernel which is been worked on, is a good thing.

I am ok with breaking compatibility when it brings a better system and new software. I am only surprised that consensus seems to be that breaking compatibility would be general a good thing. I remember very good that people not wanted to use AROS because there is no 68k integration like Petunia and it not "feels" enough like Amiga.

In my view the break between classic and NG should have happened with day one and not inheriting many of the limitations of classic amiga. Now it is already late for it. And it is the question if the development resources are there.

@OlafS25

There ways to do this with out it feeling sluggish, or alien.

https://www.youtube.com/watch?v=DUgzXX2_aDQ

We can trick old programs to see different sets of libraries..
or we can use OS4 interface version system, so that new programs use newer API’s and older programs use wrappers. We don't need to remove 68K support its maybe not needed.

Just wondering how others use E-UAE.

I emulate an Amiga classic with AmigaOS3 on my OS4 system.

I use a very old version of E-UAE.

I run the emulation in fullscreen mode, so the "classic" Amiga Workbench appears to take over the entire screen. The Workbench screen opens with a RAM disk and Workbench disk. The Workbench disk contains drawers just like on a classic AmigaOS3 system.

I have three different configurations:
- an Amiga A2000HD running AmigaOS 3.1
- an Amiga A2000HD running AmigaOS 3.2.3
- an Amiga A1200 with UAEGfx running AmigaOS3.2.3

I set up the first one many years ago. It has alot of software from my original Amiga A2000HD computer.

The second one has AmigaOS 3.2.3 and some of my old favourites.

The third one has GlowIcons and a higher screen resolution but uses a huge amount of RAM on my host OS4 system.

So basically, I am running an AmigaOS3 Workbench on top of my OS4 system. I know how to mount adf files as floppy disks, but so far I have not required access to my printer or my network from inside the emulation.

---

Some of the comments above appear to suggest another way to use E-UAE.


---
redfox

@redfox

No, we talking about running AmigaOS4.1 on top of AmigaOS4.2, or some kind of VM / Sandboxing.. like docker.

Whats new in latest EUAE for AmigaOS4.1 has support for native memory copy, and bsdsocket emulation, far easier to select configurations.

the UAE for A600GS / A1200GS, has extensions to ARM host library, and many parts of the system runs on it take advantage of that.

JanusUAE for AROS, integrated AmigaOS3.x applications into the AROS desktop.

@LiveForIt

Thank you. I really appreciate your explanation.


---
redfox

@all

Interesting discussion. My first reaction to this thread's title was "of course not!" However, I see that the discussion is much more nuanced.

Adding essential modern OS features will definitely need a backward-compatibility break. But, that doesn't mean that we can't maintain the ability to run old AmigaOS software via a sandbox of some sort.

I put some thought into how this could be done a while ago, and recorded it in my "Saving AmigaOS" series. I see some similar ideas being thrown around here. The "Saving AmigaOS" series is part of the Kea Campus Archive, which is available here.

Hans

The definition of "68K support" has meandered over the course of this thread, but it appears to come down to Exec's current design holding back SMP and introducing security and stability issues. I'm not currently much of an Amiga developer, so can't get much more technical than that. From what I recall of the Hyperion "legal papers" posted to another site, their mandate was originally a quick and dirty port to PowerPC. I think the original idea was that OS5 would be along shortly (within a decade?) and this would be good enough. But there was no SMP when all this was being worked out, and here we are 20 years later.

I don't know where @LiveForIt gets his information, but if this is the current state of Amiga SMP

Quote:

then yes, something has to change. Apparently the Extended Memory API for virtual memory is quite straightforward, but even that doesn't get used. Part of me wishes this could be put off until we've done everything once -- hello new sound API, new printer API, Unicode, NGFS for everyone, etc.! But again, here we are.

I'd love a fancy system using a real hypervisor (would like to try Linux NixOS sometime), but the quick and dirty solution this time does appear to be further integrating EUAE. I wonder if the interest in this thread mirrors A-EON's and Hyperion's priorities? OS 4.2 without real SMP would be a great disappointment.

@tao

forget DE/AA OS5, it was Amiga Inc project, that is out of the picture..

Not a lot information has shared about sate of SMP in AmigaOS4.1..
so we left guessing...

The fundamental issue for SMP is that core do not see the same content, etch core has unique L1 cache.. when data is shared between the core, the L1 cache, most be synchronized.

https://www.youtube.com/watch?v=r_ZE1XVT8Ao
https://www.youtube.com/watch?v=7yrK_9PderQ

protocol-driven cache coherence, has a lot issues.
1. Lack of Scalability (The "Bus Traffic" Problem)

Snooping Protocol Limits: Common hardware protocols often rely on "snooping" (broadcasting) to inform other cores of changes. This works well for a few cores, but as the number of cores increases, the traffic on the interconnect bus becomes a massive bottleneck, slowing down all processors.

2. High Performance Overhead (Latency and Bandwidth)

Coherence Misses: In a multiprocessor system, if multiple cores constantly write to the same data (false sharing), the hardware must invalidate the cache lines in other cores. This leads to high cache miss rates, which are far more costly than simple memory latency.

3. Inflexibility and Inefficiency

the basic issue you can do protocol-driven cache, but it inefficient, because flushes caches too often, you want the OS to control this..

this comes down to ownership of the data, what core has accessed what data, you don't want the cache to be flushed when its not needed, but you do want to flush it, if you most do so. so keeping tack of what core, did what when. to what data..

"to what data" part is the issue in AmigaOS..

Its up to AmigaOS programs to snoop public lists, because there is lack of locking mechanisms, that specify what data, that is locked, we often have to use the most powerful tool to get ownership, by disabling multitasking, for short periods. Instead of proper lookup functions. Lets say I want to obtain a task or process structure by pid, without blocking the OS, and without risk of reading something that might be freed from memory, without notice. Maybe I want to obtain a msgport by name. There is function called FindPort but does not guarantee ownership or shared lock. The msgport can be deleted without notice. unless you prevent other task form running.

@LiveForItQuote:

Interesting video, thanks! I could follow and more or less understand up until AWS Lambda, then I was lost.

@OlafS25

Whenever a subject of this nature is brought up in any Amiga forum, I remember all of the doom-and-gloom in the Amiga magazines around 1989-1990 concerning Workbench 2.0 being incompatible with around 60% of the Amiga software at the time. When 2.0 came out, I upgraded. I eventually got a kickstart switcher to run old games. I found the upgrade worth it even though it introduced some incompatibility. If AmigaOS 4.3 or higher breaks compatibility with 68k, there will be workarounds. The new features will be worth the change in my opinion.

@LiveForIt

Quote:

Of course. Just musing over "how we got here".

Quote:

Thanks for the primer. I'm glad to hear that you're guessing, as hopefully the kernel developers came up with something much more "drop-in". It apparently largely works, too. When the technical writeup finally appears (wiki.amigaos.net ?) I'll be curious to see which solution they chose.

@tao

but if we are talking about 64bit, we don't need to guess. then we must have a sandbox.. so its just over the horizon anyway..

and frankly disabling multitasking to access a public stricture is not good idea... and even worse on SMP system..

Sure maybe solve the messaging API, but AddTail() is a macro, there if you have no function that you retro fit any cache flushes, then the pre-compiled code can't become SMP friendly.

for more info look here, also notice the warnings, and bad advice.

http://amigadev.elowar.com/read/ADCD_ ... cs_3._guide/node01E1.html

you will need to do it on forbid / permit, but that is already pretty bad.

@LiveForIt
Quote:AddTail() is an exec function, in all versions of AmigaOS.

There are macros for all list functions (or at least were in some old m68k C compiler includes), but software using them instead of the exec functions usually does even much worse things as well, for example using (struct ExecBase *)SysBase->ThisTask instead of IExec->FindTask(NULL), and therefore will never work with SMP.

@joerg

Quote:

Easy... set SysBase->ThisTask to 0xdeadcafe or something silly.

put a MMU protection on 0xdeadcfae, the exception routine calls IExec->FindTask(NULL); or something like that.. (might need to find correct the IExec sense that might be a different instance, per core)

"Per-Core Structure: Almost all modern multi-core processors have an MMU integrated into each individual core. This allows each core to independently translate virtual addresses to physical addresses for the thread or process it is currently executing."

So I guess you need to set that exception 4 times, if you have 4 cores..
you might perhaps have a slight different exception per core, that handles instance of IExec..

@LiveForIt
Quote:Wrong, you'd have to unmap the page containing SysBase->ThisTask (+4092 other bytes of struct ExecBase) instead and use exception handlers to return the required data per core.

Just like there is no SysBase pointer in 0x4 but the zero page is unmapped and an exception handler returns the SysBase pointer if something is accessing 0x4 with a 32 bit read.

No big problem for the zero page, only old m68k code accesses 0x4 to get SysBase, and most only once at program start. For PPC native software it's passed as one of the _start() arguments, or other functions like in libraries, etc., but it's always available without having to access 0x4.
But if you'd unmap a large part of struct ExecBase and use exception handlers instead to return the data you'd get extreme slowdowns, it might even be slower using SMP with 2 cores with such a method than only using a single core

@joerg

with a sandbox approach you most likely end up width a different execbase anyway..

you need a ready list / wait list per core, and I think old linked list is a bit slow due to its scattered data, and its slow to index, as you need to walk the list up and down.

Quote:

page zero is already memory protected, if not you won’t get a DSI error on NULL pointers..

I have used exceptions in NallePuh, and I must say it works far better than I expected even on busy loops, pocking the same address repeatedly. No doubt is slow, but once you read address 0x00000004, into a register, you no longer reputedly poke that address.

@LiveForIt

On AROS there is a experimental version with SMP support. As I understand it the concept is one core runs the system processes, apps are automatically started on other cores or can be started on a own core. Advantage, system is more stable and expeciaslly more resposnsive

But this of course needs changes in exec that affect compatbility. No problem if your software base is limited and mostly sources available and adaptable, a problem if most software is closed and cannot be recompiled. The same is true for 68k.

Another topic is migration to 64bit. You have software that is 32bit and cannot be reompiled. Solution is a runtime where 32bit software works on 64bit system..

68k can be solved by UAE of course, for example Amiberry.

So in short you need to modernize to drop compatibility to the old 68k amigaos and also to the existing PPC base and solve it by runtime/VM for PPC software (QEMU?), perhaps runtime for closed 32bit software, integration of 68k using UAE (as transparent as possible). Petunia will no longer work

Cleaned up some weird stuff in this thread.

Stay away from the crazy stuff please, and continue the interesting discussion you're having here.

@OlafS25

I can’t comment on what is not official or is not released..

I can merely comment on issue with SMP, and cache concurrency, from the information, I looked up or find lectures about it..

if it’s possible or not to do x/y/z, comes down to problem solving abilities, and what solution was chosen. Something can be hard to do, when you have a lot of stuff running in background tripping all over the place.

So far from what I have lookup, it’s normal for cores to have their own MMU, so a exception should not affect other cores. something some people suggests that’s not case, but as pointed out by Gemini, it won’t be possible for threads to have its own virtual memory maps if that was the case.

cache concurrency has hardware solution, and software solutions, there are different ways this can be implemented, as outlines in YouTube lectures I have pointed to..

I can’t comment on the AROS solution.

If it is possible or not to run Petunia inside a sandbox, my guess is that’s its possible, however I do not know implementation, so I can’t say for sure..

I can say for sure that it is not possible to add 64bit support without a sandbox for 32bit programs..

My opinion about AmigaOS4.1 API is that is fragile and can need improved security, regarding stack, API’s and structurally.. and that major changes can require a sandbox solution.

I guess that a lot of AmigaOS libraries are not cache concurrency safe, and might need to be sandboxed. its far better if programs fail to open a library, then bring down the system