Who's Online |
92 user(s) are online ( 25 user(s) are browsing Forums)
Members: 0
Guests: 92
more...
|
|
|
|
|
Re: IBrowse 3.0 released
|
Posted on: 11/22 18:14
#1
|
Just popping in 
|
@daveyw Quote: daveyw wrote:BTW, had a slight hiccup when I upgraded from 2.5 to 3.0. The installer detected the version bump, and created a new directory for 3.0 and migrated files across. But I had some custom buttons that I had custom images for. These sit in the same dir as the other button images but for some reason were not migrated. So when I first started 3.0 I got a bunch of missing images error messages. Took me a few moments to figure out what went wrong. Installer should copy across the entire directory including any images I have added. Yes, I did ponder heavily over what to do in cases like this - I should probably have mentioned in the installer that you'll need to copy over any custom images manually, just as you installed them originally. The thinking was that some users may either not have installed any custom images or may have a load of junk in there. Those who upgraded their installs from a 2.3 or earlier install, this will be especially true - they might even have the old pre-2.4 imageset installed still (it bugs me when I see people posting screenshots with those old images, which we already changed 17 years ago!). If this is the worst thing to go wrong with the 3.0 release, I'll be very happy  . Certainly, this is something I can revisit for 3.1, to see if I can come up with a better solution, although obviously it is too late for those who already upgraded.
|
|
|
|
|
|
Re: IBrowse 3.0 released
|
Posted on: 11/20 12:39
#2
|
Just popping in
|
@daveyw Quote: daveyw wrote:
So does this mean IBrowse 2.5 dev is ended, and all future work will just be on 3? Yes - it is time for a fresh start after 23+ years, don't you think? . The original plan at the time (20 years ago!) was to be merging IBrowse 2.4 back onto the IBrowse 3.0 development branch. But the 2.4/2.5 branch diverged so much, and in some ways superceded the old 3.0 code, I had to do things the other way around - I manually merged all the missing/relevant changes from 3.0 onto 2.5.8 (2.5.9 being an unforseen emergency minor update), along with completing the new HTML parser (which Stefan had started, but only really the framework design was done). With 2.5 development, I had become a little bit scared to do too many big changes in 2.5, since I knew it would make merging with 3.0 infinitely worse. Now there is a new unified baseline. The new HTML parser was written with CSS in mind, and the plan is to further adapt this to support CSS (which means big changes to the layout and rendering engine too, amongst other things), which will hopefully be added incrementally in future 3.x releases. Some work was already done for CSS, but the code is not currently used/enabled. I know CSS was planned, 20 years ago, for the original 3.0 release - that is why I decided to make 3.0 a free upgrade for those who have already paid for the full 2.5 release.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 11/13 21:39
#3
|
Just popping in
|
@kas1e Quote: It looks like then that elf.library have 2 mechanism of binaries loading : one for pure binaries (where it possible to have .text and .rodata in one segment) and one when you load .so, where it not possible to have .text and .rodata in one segment. That correct ? 2 different pieces of code for 2 different scenarios ? The first thing that happens is that the program headers are loaded and normally each segment is then loaded in its entirety. Only if an executable segment contains a .rodata section will that segment be handled differently (the segment is not loaded, but instead it is deferred - when a section from that segment is required, that section is then loaded into a separate block of memory). It had to be done that way for all the reasons given. Quote: But then, why there were 2 different ways done at all ? For what benefits ? Why not make .rodata be separate for both types of binaries, and be done with it ? Simple code, simple understanding, everything the same for both types, no "wtf". Or there is some real benefit to have 2 different ways of handling .rodata thing ? @MightyMax Quote: So the best solution would be to always have .rodata section in its own segment ,
regardless of the type (executable, sjobs). This the elf.library allocate straight a head memory as described by the segments. No fancy splitting needed. And sjobs works, too. Yes, probably - maybe there is a good reason as to why not, but from first glance I can't see why it would cause any harm, other than increasing the size of the binary, a little. Why did the old binutils not do this? You'd have to ask whoever decided that. Probably nobody thought this would be an issue at the time, when OS4 did not support sobjs at all. As I mentioned, the 68k cross calls in early code will have been put in .data (as that's what fdtrans originally did) and if it stayed that way there wouldn't have been any of this mess with .rodata. Presumably switching from asm to C for the cross calls influenced them ending up in .rodata instead of .data (mimicking what idltool had been doing since day one). Again, at that time (2005) probably sobjs did not exist in OS4, so nobody could foresee this being an issue. Obviously, there will never be getting away from elf.library handling .rodata in 2 different ways, as old binaries will always have to be supported, no matter if newer ones put .rodata in a different segment.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 11/12 12:08
#4
|
Just popping in
|
The M68k->PPC cross calls that fdtrans generates used to be asm only, and old versions put them in .data, but this was changed to .rodata at some point. Later an option was added in fdtrans to generate them in pure C (idltool can also be use to create the same) with the cross calls declared static const, so those end up in .rodata too.
@kas1e
1) No
2) It doesn't - the .rodata issues affects all binaries. It's just that the solution used for non-dynamic objects cannot be used for dynamic objects (otherwise 24-bit PC-relative jumps in .plt could be out of range, resulting in ISI crashes, which was the case with some of alfkil's elf.library changes, which I have since fixed - I experienced this with the example code that you sent me, which can manifest itself as seemingly random crashes depending on whether .plt ended up in memory in range of .text or not).
3) Yes
What Joerg says is correct. It does indeed matter that .rodata must be allocated into non-executable memory in all cases, whereas .text (and .plt) must go into executable memory. This is not GCC specific and this is what elf.library ensures. For static objects, it doesn't matter much about .rodata, as elf.library will not load any segment containing .rodata (each section is allocated to a separate block of memory, to ensure .rodata is not placed in executable memory). For dynamic objects, that solution will break .text/.plt interaction, hence .rodata must be kept separate from .text/.plt in this case.
With VBCC, it can generate mixed PPC/68K binaries (e.g. for WarpOS), but if you're using the m68k VBCC to compile the m68k code, obviously VBCC knows which code is which and the object formats are different of course. Vlink is nice because it knows about m68k, PPC, HUNK and ELF, unlike the GCC linker which is compiled for a specific target. Don't get mixed up with VBCC generating m68k code and GCC/VBCC genarating PPC code which may contain m68k code (assembly) as data (which is what EmuTraps are). VBCC will put EmuTraps in .rodata, just as GCC does.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 11/10 10:45
#5
|
Just popping in
|
@kas1eQuote: But we talk only about shared objects when talking about .rodata needs to be out of the segment with .plt, right ? When it's pure executable, it still can be together with .text segment with no problems then (like it was with old binutils). That's correct, yes. For the reasons that @joerg has outlined, the special handling of .rodata by elf.library cannot be changed. Quote: But this behavior (having .rodata together with .text in one segment) seems to be varied from different binutils versions: sometime it was with .text, sometime in some version it was out (we found some google links where ppls curious about). For normal executables, I have only ever known .rodata to be put together in the same segment with .text, at least with all the native official SDK compilers (and even all the unofficial adtools ones I have used), and least from 4.0.3 and up. Famous last works, but it probably won't do any harm if .rodata is separated from .text in this case too. For shared objects, it may be different - it's perhaps at least partly the reason why there were two versions of shared objects (I don't know the details, but I don't think PLT worked in the old versions - the old version has been obsolete for quite some years).
|
|
|
|
|
|
Re: gcc 9 and 10
|
|
Just popping in
|
@joerg Quote: The m68k cross calls probably depend on IExec->IsNative() returning FALSE for the m68k code, just like it's required for utility.library hook functions for example checking if the hook function is PPC native or m68k code.
If .rodata is in the same segment as .text .rodata with the m68k code may be put into executable memory (segment registers on G2/G3/G4 CPUs) making it fail by IExec->IsNative() returning TRUE for m68k cross call code. I guess we are talking purely in the m68k->PPC direction, in which case the only situation an ELF binary contains any m68k code is from the EmuTrap stubs as generated by idltool/fdtrans, etc (fdtrans used to put these in .data, but now puts these in .rodata). I'm not sure where Petunia fits into things, as I couldn't see any code related to this at first glance, the emulation in the kernel performs a check similar to IExec->IsNative(), so I think you're right. Any OS3 code will fail to work on OS4, unless the EmuTraps in OS4 libraries/devices have been loaded into non-executable memory. Not much to be done about this, I guess. In theory, you could load the entire .text/.rodata/.plt segment in one block of memory and then change the memory attributes for the .rodata section, but I don't think it can be guaranteed that there is suitable padding/alignment either side of .rodata, particularly with the older binutils. That said, this could be checked and handled differently if alignment is correct, but perhaps not worth the effort.
|
|
|
|
|
|
Re: gcc 9 and 10
|
|
Just popping in
|
@kas1e Quote: kas1e wrote:
Previous you note that:
Quote:
Yes, .rodata must not be in the same segment as the .text/.plt segment.
Can you please double recheck that ? I mean, what the reasons for and why it should be moved out ?
I can understand why .plt should be the final one (same as .bss) probably, but why .rodata should be not in the same segment ? It can be just before .plt, etc, in same write protected segment ?
Or there something special about in elf.library about ? Did I not already explain it? I hope I did, but perhaps I didn't . I've checked the sources again, as I forgot why... It doesn't matter which order .text and .plt are in at all, just as long as they are both in the same segment - elf.library will load this segment as-is. If you put .rodata in the same segment as .text/.plt, then this triggers elf.library to load .text/.plt/.rodata individually into separate memory blocks (this will break the PLT) rather than loading the complete segment into a single memory block. The logic is that any segment containing .rodata is treated differently by elf.library - each section is loaded individually, rather than the entire segment. The reason: something to do with not breaking 68k cross calls, which is the only clue given in the comments (I haven't figured out exactly why this is, only that the OS crashes if I remove this behaviour). If at all possible, I'd like to change this, but until I understand it, I can't touch it.
|
|
|
|
|
|
Re: Updater tool: latest releases and updates
|
Posted on: 10/27 14:52
#8
|
Just popping in
|
@Maijestro Quote: he he yes you are of course right the updates were provided via both updater (Ehancer Software) and AmiUpdate, it is indeed a bit confusing. I assumed that both tools use the same update server. Thanks for the hint. AmiUpdate is officially supported by AmiSSL - AmiSSL updates go live on AmiUpdate within minutes of me releasing an update
|
|
|
|
|
|
Re: can anybody explain about current situation with BREL Relocations ?
|
|
Just popping in
|
@kas1e
There's no such thing as as simple test case for this, but I can come up with something when I have time. A proper test case will need to make use of IElf->CopyDataSegment() to fully cover all eventualities.
As mentioned earlier, maybe once baserel is working again, perhaps that code could be adapted to make it possible to define functions where r13 can automatically be saved/restored (like __saveds) for when using -msdata instead of -mbaserel.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 9/30 18:27
#10
|
Just popping in
|
@kas1e Thanks for your tests - glad to see the results are all as expected. It's not related to shared objects, but there is also the issue with executables where stdio in destructors is not working in newlib (works in clib2). I have reported this bug, but as nobody fixed it yet, I tried to myself and haven't figured out how to compile it so far. I'm fairly sure a public elf.library update will happen without too much trouble - it was just that it wasn't impossible to do so with the bugs added in v53.35-53.42 (and yes, one bug was my doing  ). Ideally, I "just" need to clean up the reloc routine by rewriting it - Alfkil's pointer equality fix, for example, results in an unnecessary duplicate dynamic symbol lookups when processing solibs. And then there is the bolted on symbol cache that was added at some point, which is now not used everywhere it could/should be. Performance should already be better with elf.library since v53.45 (for example, Odyssey loads here in 5 seconds, instead of 7).
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 9/30 11:27
#11
|
Just popping in
|
I released a new beta version of elf.library (53.49) yesterday, which should now behave properly if st_value is 0 or not, for undefined functions. This doesn't include my rewrite of all the reloc code, as I've not completed that yet.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 9/28 19:13
#12
|
Just popping in
|
@MigthyMax
Where can I find the old binutils elf32-amigaos.c file which has that dodgy patch in it? I need to read the surrounding code to understand exactly what areas it will affect, so I can handle it correctly in elf.library.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 9/28 11:27
#13
|
Just popping in
|
@kas1e
I think you are getting mixed up. There are two different situations - for executable programs, ctors/dtors are run by the clib2/newlib startup code - elf.library is not involved in that.
What Alfkil added in elf.library was for shared objects only - elf.library used to call __shlib_call_constructors and __shlib_call_destructors for shared objects, which in turn ran the ctors/dtors. Now elf.library doesn't call those functions and reads .ctors/.dtors itself and calls the functions directly. There is no startup code in a shared object as such (compared to an executable).
The .init_array/.fini_array handling in elf.library is again only for shared objects. For executables to support those sections, the startup code in clib2/newlib needs to be modified, as Joerg said.
|
|
|
|
|
|
Re: gcc 9 and 10
|
|
Just popping in
|
@MigthyMax
Thanks for that. It is just more difficult to understand because elfpipe's fix was not done correctly and was partially broken, causing seemingly random crashes.
I've adapted my fix to ignore R_PPC_REL24 relocs when st_value is 0, because it was originally ignoring R_PPC_PLTREL24 (for ldso version 1) when in fact those relocs do need to proceed as they originally did - this stopped Timeberwolf from crashing. I'm still not sure elfpipe's fix is quite right as it leads to duplicated lookups, but I'm hoping to resolve that if I can rewrite and cleanup the whole reloc handling routine.
Regarding binutils, you are correct that changing the code to zero st_value, as it should do, will be incompatible with earlier elf.library releases. It is a bit annoying that this "fix" was placed there, instead of fixing elf.library. The plan is to push an elf.library public update out once all reported issues have been resolved.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 9/27 14:00
#15
|
Just popping in
|
@MigthyMax
Ok. The one thing I always have to check is that Timberwolf still starts up and unfortunately it is now crashing with the change I made regarding st_value. Need to do some more testing and checking as to why this is the case.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 9/27 13:50
#16
|
Just popping in
|
@joerg Just to be clear, I wasn't the one who made those changes . However, elf.library only calls constructors/destructors directly for dynamic libraries - not executables. It does this only instead of using the old __shlib_call_constructors/destructors mechanism.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 9/26 21:53
#17
|
Just popping in
|
@kas1e From what you say, regarding the different variations, I'd have to agree that it is going to be the new executable that is the problem. The crash is PLT related - either the PLT not being updated or the jumps in the code not reaching the PLT. The obvious differences - the old binutils test_dyn:
Relocation section '.rela.plt' at offset 0x468 contains 3 entries:
Offset Info Type Sym.Value Sym. Name + Addend
01004048 00000315 R_PPC_JMP_SLOT 01004048 hello + 0
01004050 00000415 R_PPC_JMP_SLOT 01004050 memcpy + 0
01004058 00000e15 R_PPC_JMP_SLOT 01004058 gettimeofday + 0
Relocation section '.rela.text' at offset 0x1a334 contains 211 entries:
Offset Info Type Sym.Value Sym. Name + Addend
01003998 0000380a R_PPC_REL24 01004048 hello + 0
Symbol table '.dynsym' contains 21 entries:
Num: Value Size Type Bind Vis Ndx Name
3: 01004048 0 FUNC GLOBAL DEFAULT UND hello
4: 01004050 0 FUNC GLOBAL DEFAULT UND memcpy
Symbol table '.symtab' contains 99 entries:
Num: Value Size Type Bind Vis Ndx Name
56: 01004048 0 FUNC GLOBAL DEFAULT UND hello
57: 01004050 0 FUNC GLOBAL DEFAULT UND memcpy
Compared with the new binutils:
Relocation section '.rela.plt' at offset 0x468 contains 3 entries:
Offset Info Type Sym.Value Sym. Name + Addend
01004048 00000315 R_PPC_JMP_SLOT 00000000 hello + 0
01004050 00000415 R_PPC_JMP_SLOT 00000000 memcpy + 0
01004058 00000d15 R_PPC_JMP_SLOT 00000000 gettimeofday + 0
Relocation section '.rela.text' at offset 0x25584 contains 211 entries:
Offset Info Type Sym.Value Sym. Name + Addend
01003998 00003a0a R_PPC_REL24 00000000 hello + 0
Symbol table '.dynsym' contains 21 entries:
Num: Value Size Type Bind Vis Ndx Name
3: 00000000 0 FUNC GLOBAL DEFAULT UND hello
4: 00000000 0 FUNC GLOBAL DEFAULT UND memcpy
Symbol table '.symtab' contains 99 entries:
Num: Value Size Type Bind Vis Ndx Name
58: 00000000 0 FUNC GLOBAL DEFAULT UND hello
59: 00000000 0 FUNC GLOBAL DEFAULT UND memcpy
Not sure what is "correct", but it is probably related to https://www.amigans.net/modules/newbb/ ... id=136903#forumpost136903. I think this is kind of linked to Alfkil's pointer equality "fix" in elf.library. I had to undo some of that for R_PPC_REL24 relocs because his change caused crashes due to out of range 24-bit branches. So, you might find your test doesn't crash with elf.library 53.46 (will crash with newer versions). Likewise if you re-add the patch that stops st_value being set to zero, it shouldn't crash. However, I'm not sure any of that is a proper solution, so the big question is what is the correct solution? I'm happy to adjust elf.library, but how... Ideally, which I tried the other day, is that R_PPC_REL24 relocs could be ignored because the relative branches are already in the code usually (however, I found some cases where the reloc needed to be applied - newlib calls in some libraries). I suppose elf.library could be changed to ignore R_PPC_REL24 relocs completely if st_value is 0, which might solve these crashes. Edit: Just tried my own suggestion above and your new binutils test_dyn is now working fine. If you can confirm your new binutils test_dyn doesn't crash with elf.library 53.46, it would make me a bit happier that this fix is the correct thing to do. FYI, once we can confirm that elf.library is handling things correctly and fully fixed, I'm going to bump it to v54.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 9/21 15:58
#18
|
Just popping in
|
Sorry for the delayed reply - have been busy attempting to rewrite the reloc code in elf.library, so that it's easier to understand and less easy to break in the future, which might not be possible! @MigthyMax This looks like a linker issue to me - segment 2 and 3 overlap, which shouldn't happen: segment 2: 0x01000000-0x010050B0 segment 3: 0x01005000-0x010050A6 ELF.debug is levelled also - can be set 1-9 (maybe higher too). The higher the value, the more debug you get. @kas1e I don't have anything to test it with, as I don't think I have anything with a .fini_array and .init_array in them, or anything that can generate them. However, it looks like it should work fine, but the current logic involved is that it looks for .ctors first, and if .ctors is not present, it then looks for .init_array instead. Same for .dtors and .fini_array. So, if you were to have both in the file, .ctors/dtors will always get used.
|
|
|
|
|
|
Re: gcc 9 and 10
|
Posted on: 9/12 17:31
#19
|
Just popping in
|
@MigthyMax Quote:
So instead of using the program headers (segments) to allocated memory
as described by the elf file, the elf.library allocates memory by sections?
I thought that this is one of the smart ideas of the elf file format, that
a linker just allocates memory as describes and than populate it from the
content, resp. fills its as described by the ABI.
Yes, well the normal case is that elf.library scans the program headers and allocates a memory block to hold each segment. Then, if/when it needs to load a section, the section is loaded into the defined location in the segment. However, currently, segments will be ignored if that segment contains .rodata. Then, instead of the above, each section is allocated individually later when it needs to be loaded. This means even for the old binutils when .rodata is in its own segment, that segment is ignored too. For the next elf.library, I'm tweaking this slightly, so that segments will always be loaded in their entirety, unless the segment contains .rodata and an executable section (i.e. old binutils where .rodata and .text were placed in the same segment). Quote:
And you say, that is because of the support of M68k stubs stuff?
That's what the comment eluded too, but I've still yet to get my head around exactly why. I vaguely remember that in the old days the stubs, which we asm based back then, were maybe not placed in the correct section or something. Quote:
Quote:
This makes the ctors/dtors crash (as elf.library tries to execute address
0xfffffffc instead of skipping this entry), when I try to recompile your
test code here.
Mhh, that could be case why stuff crashes with new binutils. But i read
further down that this is already fixed in new elf.library. And that
ctor/dtor stuff heavily even depends on the used c library.
Certainly if you are seeing ISI errors with the instruction pointer at 0xfffffffc, or thereabouts, that will be the reason. I think the whole reason Alfkil changed elf.library to use .ctors/dtors directly, instead of relying on __shlib_call_constructors(), was to make it independent of newlib/clib2 (and to handle broken apps that had unterminated ctor/dtor arrays). It wouldn't have been a problem if the recent change to newlib had been mirrored in elf.library at the same time. The dtors not working correctly is almost certainly a bug in newlib.library and I've filed a report for that so hopefully it gets fixed soon. (Un)fortunately, I got carried away with fixing these things in elf.library and have fixed and improved some other things too. I should be working in a different project, but whilst I have all these issues and inner workings of elf.library in my head, it's best to flush them all out before I forget .
|
|
|
|
|
|
Re: gcc 9 and 10
|
|
Just popping in
|
@kas1e
Which version was the older elf.library that you were using (you quoted the newlib version instead)?
There are several elements involved here:
1. Prior to elf.library 53.37, __shlib_call_(de)constructors() was called (located in SDK:newlib/shcrtbegin.o). With 53.37 or higher, it doesn't use that anymore and instead directly reads .ctors/.dtors and executes the functions specified there.
2. No matter which binutils, compiler or newlib.library you are using, you also have to consider which version of the newlib dev stuff that you have installed in SDK:newlib/lib - for example, the new startup code supplied with 53.84 that sets the initial .ctors/.dtors entry to 0xffffffff instead of 0, which causes elf.library to crash. If you update only "newlib.library.kmod" and leave SDK:newlib/lib as some older version, it won't crash. Likewise if you use some older "newlib.library.kmod", but have SDK:newlib/lib at 53.84, it will still crash.
In my case, I only have the official native compilers and binutils installed, from SDK 54.6, including GCC 8 and 11, plus 4.0.4 and 4.2.4 from older SDKs. No matter what I use, dtor and dtor2 will not print when using newlib (latest SDK:newlib from 53.84), only clib2 works. I'm going to assume this is a bug and that the deconstructors are being called too late in newlib, so I'll file a Bugzilla ticket for that if there isn't already one there.
Regarding elf.library, I've already fixed the 0xffffffff .ctor/.dtor handling and I've tested that it doesn't crash anymore, which will be available when I upload 53.43 for beta testing.
For the new binutils, the change that is required is to ensure the .rodata section is placed in its own read-only segment by itself for dynamic objects, just as the old binutils does. It is not a good idea to put any other sections in the segment holding .rodata, due to previously discussed reasons.
Let me know if I have missed anything?
|
|
|
|
|
News feed
OS4 Feedback forum
OS4Depot Feedback forum
Amigabounty forum
OpenAmiga forum
AmiCygnix forum
ABC shell forum
AmiKit forum
Cinnamon Writer forum
CodeBench forum
Digital Universe forum
Dopus 5 forum
E-UAE forum
Gnash forum
Ibrowse forum
JAmiga forum
Odyssey forum
OWB forum
Qt forum
SmartFileSystem forum
Timberwolf forum
TouchDevice forum
TuneNet forum
Unsatisfactory Software forum
