Odyssey 1.23 progress: r5 RC2

@Petrol

I never use PThreads in my programs because, because lose some control of child process it creates, for example in mplayer.

@samo79

Yes I notices there might be problems with libcurl and having it running as child process, in fact I can't do it on LiveFroIt Music, I guess it's because of how bsdsocket api was designed in OS4 version of roadshow. I ended up writing the download feature as a non-threaded external program, to avoid that horror show.

So in my case main program creates a thread to communicate with external program, a struct with information is provided to the external program, the external program use that information in struct and updates it.

@Deniil

Quote:

The debug says it tries to store an int16_t value from r3 to address of r9.
And r9 was 0x4637C140, not it have been any random value, it possible that memory was corrupted before grim repair detected it. As I have learned it far from built prof.
This thing is not safe at all, not in system with minimum memory protection.

@all
Source code uploaded, check this out:

https://www.amigans.net/modules/xforum ... hp?topic_id=8179&forum=40

Also, there was some progress I do lately: I rebuild the whole beast over GCC 8.2.0 and with fixing some annoying bits I was lucky enough to make it compiles and links (that was a side effect of making preparation for GitHub upload).

Switching to 8.x GCC may give us more speed, as well as give us some warning about things that need to be fixed. It is also some kind of pre-test to porting later 1.25 version from Deadwood's repo.

Binary works, though it did bring me some scary new bug that needs to be sorted out before another beta-release.

New Issue is that for some reason paths in some parts are all trashed now. For example, PROGDIR:Conf/ for saving/loading webicons.db, have "Conf/" trashed with all weird characters of a strange size. Building the same code over GCC 4.4.3 (with some small changes not related to that issue) does not have that bug.

Visually when you start new binary, you have at first no webicons, as they can't be loaded from PROGDIR:Conf/WebPageIcons.db, and instead created a new directory of messed characters in which place new WebPageIcons.db

All I can say about the bug is that it comes from "threads". For example, there is an IconDataBase thread, which creates WebIcons.db. Or the same happens with the "LocalStorage" thread, which saves things too and which "path" is messed now.

As with web icons it easier to trace, I just checked that it loaded from there:

https://github.com/kas1e/Odyssey/blob/ ... con/IconDatabase.cpp#L926

So, it's defaultDatabaseFilename somehow trashes.

Checking then where that defaultDataBaseFilename setup, there are functions which init all that:

https://github.com/kas1e/Odyssey/blob/ ... i/WebIconDatabase.cpp#L96

So any actual help (taking into account that all can browse source code already) appreciated!

I was sure at first it's something about "athread=native", but then switching to "athread=single" (so be like old gcc 4.4.3) do not solve the issue, sadly.

@kas1e

Quote:

Sound cool :)
But how much faster and in what specific operation ?

Btw, thanks for releasing code, looking forward for new progress !

@samo
I do not know will it faster or not. Only maybe, as gcc 8.x is better than 4.x in many aspectes, and in terms of internal optimisation too.

@kas1e

Ok, after all we will have all the time to test everything :)
I understand your current plan is start working on this baserel some more and then updating to 1.25, so for sure we will have good moments for tests

@kas1e

Thank you very much, it sounds that there is progress!

@kas1e

Thanks again for working on this.

Any ideas how hard it would be to update the OpenSSL or the
Curl being used? Could it be as simple as a rebuild or
more involved?

Thanks,

PJS

@pjs

Quote:

Both are some work for sure, not just "./configure;make". For curl I remember there were issues, like when you use new curl, it didn't work as expected and changes in odyssey's code need to be done. But is there needs for newer curl at all? I mean not every time when new version out is necessary need it?

For OpenSSL, I for sure can understand to be updated, but 1.23r4 from os4depot use v1.0.1q, so not very old IMHO ? But that one should be more or less easer to build, just, as I remember, that also was issues. You may remember that pure ./configure;make wasn't enough when we made test version with it, and MickJt was in need to reconfigure it somehow special so to make all works as need it.

But yes, OpenSSL sure can be more or less easily updated, while Curl needs some work (and in odyssey itself probably too).

Now with source code in github, probabaly any developer may try to experiment with other libs. I put in SDK just those ones which i use for release versions, and which 100% works and tested in few years by users.

Hi kas1e,

Yeah, somehow I doubted it was going to be THAT easy, but
I had hope... ;D

I only mentioned Curl since I figured it might be involved
in some of all the transport security updating that has
been going on since the last update (heartbleed, etc etc
etc).

The same goes for OpenSSL... There are so many secure sites
(banks, etc) that kick out errors or just refuse to connect
now and I've wondered if security updates might help that.

Thanks again,

PJS

@kas1e

While it's not a straight ./configure && make, I didn't do anything specific for Odyssey, but just some tweaks to get it to compile on OS4. There's a document openssl-os4-howto.txt in the Documentation/ directory with notes on how to compile it.

At the time I built 1.0.1q for you, there was already a 1.0.2 series, but 1.0.2 couldn't connect to some sites while the latest in the 1.0.1 series connected to most, and if I remember correctly, some sites were moving away from an old cipher. I can see in the readme on OS4Depot that I was already building the 1.0.2 series at that time.

The 1.1.0/1.1.1 series has some API changes, but I suspect if you can tweak Odyssey to work with it, it'd be able to connect to more sites, although that doesn't help in the JS/HTML/CSS department.

@MickJT
Yeah, that what I remember too: too recent OpenSSL just fails to works with some sites when linked with Odyssey.

@All
I may try to build with very latest OpenSSL now, but first, need to deal with "why when we switch to 8.x GCC, PROGDIR:Conf" have trashed "Conf" part (and only that!), when used from other threads (as all resources loaded by Odyssey didn't have that issue, seems only those ones which load from other threads such as OWBICon or OWBLocalStorage thread). I do not know what can be the cause at the moment.

All I can say, that when we switch to 8.x GCC whole thing start to bring a loot warnings, so I had to hide them by adding "-Wno-implicit-fallthrough -Wno-ignored-attributes -Wno-unused-function -Wno-expansion-to-defined -Wno-class-memaccess", but not sure if any of them can cause that. Check CMAKE/AddGlobalCompilerFlags.cmake

Also, when we build BAL/Filesystem/WebCore/Posix/BCFileSystemPosix.cpp, it bring those errors:



Not sure if any of them can be related.


Native-amiga Threading files placed in the BAL/Types/WTF/MorphOS ,but also currently do not know if it anyhow related.

Probably good-old printf debugging should be started.

@kas1e

Spotted a problem in Source/WebKit/OrigynWebBrowser/Api/WebPreferences.cpp:


This creates a temporary copy of the string object, and then returns a pointer to its internal C string. Unfortunately, the temporary copy gets destroyed when the method returns, leaving the pointer hanging. You've been "lucky" that it happened to work with older compilers.

I haven't tried it yet, but the following should work:

Note the & in front of the value. That turns the variable into a reference that points to the original string, which won't disappear the moment the method returns.

EDIT: Looks like that same mistake was done multiple times in the same file...

Hans

@kas1e
Just tested the fix I outlined above, and it works. I'll send you a pull request on GitHub soon...

Hans

@Hans
Thanks! That was fast ! At least opening of code worth so we will have now new version builded over 8.3.0. Already better than nothing.

I will try now to build it with newer openssl version and give a test archive for everyone to check.

@Mick
Quote:

Tried to use latest openssl from os4depot, and api changes there is no big issue for Odyssey's code itself, but for CURL which is use SSL for connections.

They (openssl) no more provide with SSLeay() and related stuff, and so that version of Curl which we use (7.28.1) and which is prove-tested-works-fine can't be linked over new openssl. But newer versions of Curl are harder because it need our-custom-threading-resolver implementation as i have in 7.28.1 , so it need some work for sure.

@Hans

Tested with GCC8 and interestingly, returning a pointer to local int value seems to trigger always a warning but returning char pointer to std::string content requires -O2 for that warning to trigger.

Thanks for fixing it.

@All
Okkey ! First one since few years :)

http://kas1e.mikendezign.com/aos4/ody ... odyssey_1.23r5_beta01.lha

Odyssey 1.23 r5_beta01 - AmigaOS4

-- Source code with all amigaos4 specific changes and custom 3d party linker libraries are on GitHub now: https://github.com/kas1e/Odyssey/
-- Recompiled Odyssey with GCC 8.3.0, previously it was 4.4.3
-- Fixed dangling pointer issue with WebPreferences, which was responsible for the "Conf" directory string becoming random garbage (c) Hans de Ruiter
The issue was here for a long time, just with older GCC we were lucky enough to not step in.
-- Fixed lame crash on exit happens when you use a beta of MUI and it brings you "expire" window.
-- Application.library previously got v1 of the interface, but for some time it should be 2, fixed.
-- Updated codesets.library to the latest one (6.21, 16 Jan 2018)
-- Updated curl-ca-bundle.crt (https://curl.haxx.se/ca/cacert.pem , 01 Jan 2020)
-- Compiled with Curl 7.68.0, OpenSSL 1.1.1c & RTMP 2.4.
-- actual build date and version in "about"


Now it surely will have new issues and new bugs.

For first, i expect there will be issues with sites conneted/disconnected, https and stuff. Probabaly there maybe new issues with threaded resolver in curl (because i added it to the very newer version of curl by logic from old one, so far in my tests it seems works). As well as latest openssl can cause some issues too (as we noticed before when updating openssl few years ago).

I didn't change anything in Odyssey in that terms, just use latest versions (only do changes in Curl to have it builds over amigaos4 + threaded resolver fix + file:/// fix)

Seems google start to reacts better now, but for youtube i see some images stop loading with new curl/opensll. So somewhere something should be fixed seems so. Maybe in odyssey this time, in something like BAL/Network/WebCore/Curl/BCSSLHandleCurl.cpp or so.

In other words test it, and let's collect issues. In the worse case we always can rollback on old curl and old openssl. Through, better to fix things with new ones, as we for sure better to use latest one, if we ever consider later to made another browser on top of webkit or update core in that one.

@kas1e

Cool, but archive probably corrupted ?
Got error when extracting: "Odyssey/datas/wget"

wrong checksum