Login
Username:

Password:

Remember me



Lost Password?

Register now!

Sections

Who's Online
25 user(s) are online (18 user(s) are browsing Forums)

Members: 0
Guests: 25

more...

Headlines

 
  Register To Post  

Memory Page Locking Confusion
Home away from home
Home away from home


See User information
I noticed that there is a new Hyperion blog entry:
http://blog.hyperion-entertainment.biz/?p=705

Having read it, I want to thank Hyperion for listening & being willing to revise an earlier design decision in light of feedback.

Author of the PortablE programming language.
I love using Amiga OS4
It is pitch black. You are likely to be eaten by a grue...
Go to top
Re: Memory Page Locking Confusion
Just can't stay away
Just can't stay away


See User information
Quote:

ChrisH wrote:

I want to thank Hyperion for listening & being willing to revise an earlier design decision in light of feedback.

I'm also glad they did! Thank you.

The Rear Window blog

AmigaOne X5000 @ 2GHz / 4GB RAM / Radeon RX 560 / ESI Juli@ / AmigaOS 4.1 Final Edition
SAM440ep-flex @ 667MHz / 1GB RAM / Radeon 9250 / AmigaOS 4.1 Final Edition
Go to top
Re: Memory Page Locking Confusion
Quite a regular
Quite a regular


See User information
This quote puzzles me:

Quote:
Shared and executable memory is locked implicitly for 68K backwards compatibility reasons. Use AVT_Lock (or equivalent) to ensure your shared and executable memory is not locked.


Why on earth is executable memory locked for 68k backward compatibility when only PPC code is executable??! Seems like a complete waste of locked memory. This means that when monster apps like MPlayer, OWB and TW crashes or hangs (mostly hangs) the system can't swap out the large chunk of executable code.

Software developer for Amiga OS3 and OS4.
Develops for OnyxSoft and the Amiga using E and C and occasionally C++
Go to top
Re: Memory Page Locking Confusion
Just popping in
Just popping in


See User information
Quote:
Why on earth is executable memory locked for 68k backward compatibility when only PPC code is executable??! Seems like a complete waste of locked memory. This means that when monster apps like MPlayer, OWB and TW crashes or hangs (mostly hangs) the system can't swap out the large chunk of executable code.


Task switching is still done in an interrupt. If the new task's code is currently swapped out the scheduler would have to wait for the pages to be swapped in again. But waiting for an arbitrary long time is not possible in an interrupt and you cannot tell how long the low lever hard disk driver will need to perform the swap in job. Thus executeable code must be accessible at any time and hence must be locked to prevent it from being swapped out.

Go to top
Re: Memory Page Locking Confusion
Home away from home
Home away from home


See User information
What I have problems understanding is that this blog is taking about shard memory.
But shard memory should be available to all applications, so it does not make sense to swap it out, stack that consists many of local variables, private memory, should how ever be swappable.

(NutsAboutAmiga)

Basilisk II for AmigaOS4
AmigaInputAnywhere
Excalibur
and other tools and apps.
Go to top
Re: Memory Page Locking Confusion
Home away from home
Home away from home


See User information
@Daniil
Quote:
OWB and TW crashes or hangs (mostly hangs) the system can't swap out the large chunk of executable code.


if you just thinking freeing physical memory, that might work, but you might have a slowdown if the program counter jumps in to swapped memory.

There are lots of API’s that depends on hooks, and there are lots of pluggins that depends on program code being sheared, it might not be performance vice smart having that memory being swapped in and out.

(NutsAboutAmiga)

Basilisk II for AmigaOS4
AmigaInputAnywhere
Excalibur
and other tools and apps.
Go to top
Re: Memory Page Locking Confusion
Home away from home
Home away from home


See User information
@LiveForIt Quote:
But shard memory should be available to all applications, so it does not make sense to swap it out

Swapping memory out does NOT prevent it from being accessed (except from an interrupt). Therefore paging out shared memory is perfectly fine, since it can be swapped back in automatically.

Author of the PortablE programming language.
I love using Amiga OS4
It is pitch black. You are likely to be eaten by a grue...
Go to top
Re: Memory Page Locking Confusion
Just popping in
Just popping in


See User information
The problem with memory paging in AmigaOS is that nobody can really tell what the memory is used for, and when it will be accessed. So backward compatibility dictates a rather "conservative" memory locking, basically, everything that is not implicitly marked as available for paging has to be locked in memory.

Code is sadly one of those, since you can never know what code can be executed in what context. If code is run in a task context other than the disk devices and the pager itself, it could be paged out, but as soon as code is executed in a environment that does not allow waiting, it can not be paged out.

Having said that, we are working on "fixing" this situation. For example, we have plans for running interrupts like tasks, i.e. an interrupt from a hardware would basically just signal a task that such an interrupt occurred and the interrupt handler code would run in a high priority task. That way, it *can* wait and thus would be able to wait for paging operations.

This is just an example, of course, there are other problem areas, but rest assured that we are working on those.

Go to top
Re: Memory Page Locking Confusion
Just popping in
Just popping in


See User information
Quote:
There are lots of API’s that depends on hooks, and there are lots of pluggins that depends on program code being sheared, it might not be performance vice smart having that memory being swapped in and out.


The pager favors old data for page-out operations, so memory that is frequently accessed, or (in theory) code that is run often wold be relatively "hot" and thus would be unlikely to be paged out.

Go to top
Re: Memory Page Locking Confusion
Just popping in
Just popping in


See User information
Quote:
What I have problems understanding is that this blog is taking about shard memory.


I think that is an area that is prone to misunderstanding, that's true. Most people associate shared memory with locked memory, and private memory with unlocked (page-able) memory.

However, there is no connection between shared/private and locked/unlocked memory, only the fact that shared memory is by default allocated as locked. This is simply because most of the time, shared memory is assumed to be used for things like system structures, messages etc, everything that has to be accessed by multiple processes.

Private memory, on the other hand, is supposed to be accessed only by the application allocating it. As such, it is allocated in an unlocked state by default since the MEMF_PRIVATE has to be explicitly given, and the application is supposed to "know what it's doing".

Currently, these two types of memory are also only semantically different, they are, however, not really different in the sense that even now, MEMF_PRIVATE could be accessed from other processes as well (although it would be considered a programming error), simply because the system does not yet offer real per-task memory contexts. This will change in the future though.

Go to top
Re: Memory Page Locking Confusion
Just popping in
Just popping in


See User information
Quote:
Why on earth is executable memory locked for 68k backward compatibility when only PPC code is executable??


This is a bit misleading in it's wording. It does not have anything to do with 68k code, but rather the fact that no program (especially not programs that are basically ports from OS3) has the ability to flag parts of it's code as interrupt executable. Therefore, any ode has to be assume interrupt executable (the other way around would be problematic).

So it's not 68k compatibility, but rather backward compatibility in general.

Go to top
Re: Memory Page Locking Confusion
Just popping in
Just popping in


See User information
Quote:
Task switching is still done in an interrupt. If the new task's code is currently swapped out the scheduler would have to wait for the pages to be swapped in again.


Actually, that's not quite true. The scheduler just restored the task context, it does not bother about the tasks code. It does not access the actual task's code. So, assuming that a task is rescheduled and the task's current code is paged out, the scheduler would just load the context, and restart the task. Of course, the newly launched task would then not find the code in memory, would page fault, which would cause it to be suspended again and cause a page-in operation on the page that was to be executed. Only then would it launch successfully.

The same is basically true for any task that is launched by the scheduler that was interrupted on a page fault that has not been served up to then, data or code doesn't matter.

Go to top
Re: Memory Page Locking Confusion
Amigans Defender
Amigans Defender


See User information
Quote:
What I have problems understanding is that this blog is taking about shard memory.

My blog entry also tries to point you to the full explanation of the memory subsystem which is on the wiki at http://wiki.amigaos.net/index.php/Exec_Memory_Allocation

Please review the information on the wiki. Don't depend on my single blog entry to tell you the full story.

If anything is unclear on the wiki then please let us know. The goal here is to make the wiki your only stop for factual information.

ExecSG Team Lead
Go to top
Re: Memory Page Locking Confusion
Just popping in
Just popping in


See User information
@SSolie

I have one question: Do you plan to make AmigaGuide or HTML format version of Autodocs from Software Developer Kit for AmigaOS 4? For now they are only in plain text format. Or can you give permission to place the documentation in such format? Or the Wiki is to be the only place where the updated documentation is placed?

Go to top
Re: Memory Page Locking Confusion
Amigans Defender
Amigans Defender


See User information
Quote:
I have one question: Do you plan to make AmigaGuide or HTML format version of Autodocs from Software Developer Kit for AmigaOS 4?

Yes of course. However, I don't consider it a priority. The real meat of the documentation is what is not in the autodocs and includes. That is the stuff people also don't want to write because it is difficult.

You can find a copy of the autodocs online today at http://zerohero.homeip.net/~zerohero/autodocs/os4/

My goal is to automate the generation of the autodocs and have them uploaded to the server automatically. When that is working you'll find them on the official site as well. But again, that is the easy part.

ExecSG Team Lead
Go to top
Re: Memory Page Locking Confusion
Amigans Defender
Amigans Defender


See User information
Quote:

tfrieden wrote:
Private memory, on the other hand, is supposed to be accessed only by the application allocating it. As such, it is allocated in an unlocked state by default since the MEMF_PRIVATE has to be explicitly given, and the application is supposed to "know what it's doing".


Is it OK to allocate MEMF_PRIVATE memory, and then pass it to a library? I've never been quite clear on whether libraries are classed as separate processes.

Go to top
Re: Memory Page Locking Confusion
Home away from home
Home away from home


See User information
@Chris

Quote:

Is it OK to allocate MEMF_PRIVATE memory, and then pass it to a library? I've never been quite clear on whether libraries are classed as separate processes.


Libraries are just code. They run on the task / process that caled the code. so something MEMF_PRIVATE could be passed to the library as it's being called in your own task/ process.





Go to top
Re: Memory Page Locking Confusion
Home away from home
Home away from home


See User information
Quote:
so something MEMF_PRIVATE could be passed to the library as it's being called in your own task/ process.

Unless of course the library decides to spawn it's own task/process, such as I imagine the asyncio.library does... But I guess this isn't very common, and arguably it should be up to the library to either document this requirement or (better) copy the supplied data into memory that is sharable.

Author of the PortablE programming language.
I love using Amiga OS4
It is pitch black. You are likely to be eaten by a grue...
Go to top
Re: Memory Page Locking Confusion
Amigans Defender
Amigans Defender


See User information
Quote:

broadblues wrote:
Libraries are just code. They run on the task / process that caled the code. so something MEMF_PRIVATE could be passed to the library as it's being called in your own task/ process.


OK, cool, that's what I thought (otherwise MEMF_PRIVATE memory would be almost useless) but good to have it confirmed.

Go to top
Re: Memory Page Locking Confusion
Just popping in
Just popping in


See User information
Quote:
Is it OK to allocate MEMF_PRIVATE memory, and then pass it to a library? I've never been quite clear on whether libraries are classed as separate processes.


As has been said, MEMF_PRIVATE means private to the process. If it is not accessed outside of the process, it's OK.

Go to top

  Register To Post

 




Currently Active Users Viewing This Thread: 1 ( 0 members and 1 Anonymous Users )




Powered by XOOPS 2.0 © 2001-2016 The XOOPS Project