codecvt clib2 and newlib

	Bottom Previous Topic Next Topic
Register To Post

SinanSam460

Posted on: 2022/9/25 8:49 #1

Not too shy to talk

@all

Am I doing something wrong while compiling following source with clib2 ?


#include <iostream>

#include <fstream>

#include <string>

#include <locale>

#include <iomanip>

#include <codecvt>

#include <cstdint>

 

// utility wrapper to adapt locale-bound facets for wstring/wbuffer convert

template<class Facet>

struct deletable_facet : Facet

{

    template<class... Args>

    deletable_facet(Args&&... args) : Facet(std::forward<Args>(args)...) {}

    ~deletable_facet() {}

};

 

int main()

{

    // UTF-8 narrow multibyte encoding

    std::string data = reinterpret_cast<const char*>(+u8"z\u00df\u6c34\U0001f34c");

                       // or reinterpret_cast<const char*>(+u8"zß水🍌")

                       // or "\x7a\xc3\x9f\xe6\xb0\xb4\xf0\x9f\x8d\x8c"

 

    std::ofstream("text.txt") << data;

 

    // using system-supplied locale's codecvt facet

    std::wifstream fin("text.txt");

    // reading from wifstream will use codecvt<wchar_t, char, mbstate_t>

    // this locale's codecvt converts UTF-8 to UCS4 (on systems such as Linux)

    fin.imbue(std::locale("en_US.UTF-8"));

    std::cout << "The UTF-8 file contains the following UCS4 code units:   ";

    for (wchar_t c; fin >> c; )

        std::cout << "U+" << std::hex << std::setw(4) << std::setfill('0')

                  << static_cast<uint32_t>(c) << ' ';

 

    // using standard (locale-independent) codecvt facet

    std::wstring_convert<

        deletable_facet<std::codecvt<char16_t, char, std::mbstate_t>>, char16_t> conv16;

    std::u16string str16 = conv16.from_bytes(data);

 

    std::cout << "\nThe UTF-8 file contains the following UTF-16 code units: ";

    for (char16_t c : str16)

        std::cout << "U+" << std::hex << std::setw(4) << std::setfill('0')

                  << static_cast<uint16_t>(c) << ' ';

}

Output is :


ppc-amigaos-g++ -mcrt=clib2 -athread=native test.cpp -o codecvt -lpthread -lz

/tmp/cc0nN0eZ.o: In function `_ZNSt7codecvtIDsc10_mbstate_tEC2Ej':

test.cpp:(.text._ZNSt7codecvtIDsc10_mbstate_tEC2Ej[_ZNSt7codecvtIDsc10_mbstate_tEC5Ej]+0x2e): undefined reference to `_ZTVSt7codecvtIDsc10_mbstate_tE'

test.cpp:(.text._ZNSt7codecvtIDsc10_mbstate_tEC2Ej[_ZNSt7codecvtIDsc10_mbstate_tEC5Ej]+0x32): undefined reference to `_ZTVSt7codecvtIDsc10_mbstate_tE'

/tmp/cc0nN0eZ.o: In function `_ZN15deletable_facetISt7codecvtIDsc10_mbstate_tEED2Ev':

test.cpp:(.text._ZN15deletable_facetISt7codecvtIDsc10_mbstate_tEED2Ev[_ZN15deletable_facetISt7codecvtIDsc10_mbstate_tEED5Ev]+0x30): undefined reference to `_ZNSt7codecvtIDsc10_mbstate_tED2Ev'

/tmp/cc0nN0eZ.o:(.rodata._ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE[_ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE]+0x10): undefined reference to `_ZNKSt7codecvtIDsc10_mbstate_tE6do_outERS0_PKDsS4_RS4_PcS6_RS6_'

/tmp/cc0nN0eZ.o:(.rodata._ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE[_ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE]+0x14): undefined reference to `_ZNKSt7codecvtIDsc10_mbstate_tE10do_unshiftERS0_PcS3_RS3_'

/tmp/cc0nN0eZ.o:(.rodata._ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE[_ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE]+0x18): undefined reference to `_ZNKSt7codecvtIDsc10_mbstate_tE5do_inERS0_PKcS4_RS4_PDsS6_RS6_'

/tmp/cc0nN0eZ.o:(.rodata._ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE[_ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE]+0x1c): undefined reference to `_ZNKSt7codecvtIDsc10_mbstate_tE11do_encodingEv'

/tmp/cc0nN0eZ.o:(.rodata._ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE[_ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE]+0x20): undefined reference to `_ZNKSt7codecvtIDsc10_mbstate_tE16do_always_noconvEv'

/tmp/cc0nN0eZ.o:(.rodata._ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE[_ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE]+0x24): undefined reference to `_ZNKSt7codecvtIDsc10_mbstate_tE9do_lengthERS0_PKcS4_j'

/tmp/cc0nN0eZ.o:(.rodata._ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE[_ZTV15deletable_facetISt7codecvtIDsc10_mbstate_tEE]+0x28): undefined reference to `_ZNKSt7codecvtIDsc10_mbstate_tE13do_max_lengthEv'

/tmp/cc0nN0eZ.o:(.rodata._ZTI15deletable_facetISt7codecvtIDsc10_mbstate_tEE[_ZTI15deletable_facetISt7codecvtIDsc10_mbstate_tEE]+0x8): undefined reference to `_ZTISt7codecvtIDsc10_mbstate_tE'

collect2: error: ld returned 1 exit status

Compiling with newlib is impossible:


$ ppc-amigaos-g++ -athread=native test.cpp -o codecvt -lpthread -lstdc++ -lz

test.cpp: In function ‘int main()’:

test.cpp:28:10: error: ‘wifstream’ is not a member of ‘std’; did you mean ‘ifstream’?

   28 |     std::wifstream fin("text.txt");

      |          ^~~~~~~~~

      |          ifstream

test.cpp:31:5: error: ‘fin’ was not declared in this scope

   31 |     fin.imbue(std::locale("en_US.UTF-8"));

      |     ^~~

test.cpp:38:10: error: ‘wstring_convert’ is not a member of ‘std’

   38 |     std::wstring_convert<

      |          ^~~~~~~~~~~~~~~

test.cpp:39:70: error: expected primary-expression before ‘,’ token

   39 |         deletable_facet<std::codecvt<char16_t, char, std::mbstate_t>>, char16_t> conv16;

      |                                                                      ^

test.cpp:39:72: error: expected primary-expression before ‘char16_t’

   39 |         deletable_facet<std::codecvt<char16_t, char, std::mbstate_t>>, char16_t> conv16;

      |                                                                        ^~~~~~~~

test.cpp:40:28: error: ‘conv16’ was not declared in this scope

   40 |     std::u16string str16 = conv16.from_bytes(data);

Sinan - AmigaOS4 Beta-Tester
- AmigaOne X5000
- AmigaOne A1222
- Sam460ex

kas1e

Re: codecvt clib2 and newlib

Posted on: 2022/9/25 9:30 #2

Home away from home

@Sinan
As i say in another topic, those mbstate and stuff undefs happens because you need to recompile stdc++ with new clib2. At least that what i were told by Andrea last time i asked about those undefs about mbstte. But i can't say 100% that this is the case, so you need to recompile stdc++ with new clib2 and try. If that will deal with - good, if not , then bug report to Andrea.

If i remember right, there were some script from Andrea which helps to recompile just stdc++ and not whole adtools.

Join us to improve dopus5!
AmigaOS4 on youtube

SinanSam460

Re: codecvt clib2 and newlib

Posted on: 2022/9/25 10:17 #3

Not too shy to talk

@kas1e

When I check, I can see there is libstdc++.a 24,4 MB at
usr/local/amiga/ppc-amigaos/lib/clib2

and also libstdc++.a 17,3 MB at
usr/local/amiga/ppc-amigaos/lib/

I guess I need to report Andrea ?

Sinan - AmigaOS4 Beta-Tester
- AmigaOne X5000
- AmigaOne A1222
- Sam460ex

kas1e

Re: codecvt clib2 and newlib

Posted on: 2022/9/25 10:51 #4

Home away from home

@Sinan
No no, nothing to report to Andrea.

What i mean, is that you need to recompile clib2's libstdc++ from adtools.

I.e. you do have your crosscompiler, and you build it with original clib2 coming with it (old one). Now, to have all works correctly in c++ code, you need to recompile adtools with new clib2 includes/libs/etc, so to have new clib2's libstc++ builded with new clib2 includes/libs.

In other words : libstdc++ coming with SDK are not enough, you need to recompile it with new clib2 includes/libs.

Join us to improve dopus5!
AmigaOS4 on youtube

rjd324

Re: codecvt clib2 and newlib

Posted on: 2022/9/25 11:01 #5

Quite a regular

I do have a fork from adtools: https://github.com/3246251196/adtools

which changes the makefile to use the latest clib2 (head revision at the time of running make), among other things, like fix the broken issue of private Exec Reschedule function.

kas1e

Re: codecvt clib2 and newlib

Posted on: 2022/9/25 11:35 #6

Home away from home

@Sinan
Ok tested. Rebuild libstdc++ indeed fix undefs about mbstate,etc.

So, all you need to do if you don't want to install new cross-compilers and stuff is:


cd /amiga/adtools/



rm -f ./native-build/gcc-cross-done-11.3.0



find ./native-build/gcc-cross-build-11.3.0/ppc-amigaos/clib2 -name *.la -exec rm -f {} +

find ./native-build/gcc-cross-build-11.3.0/ppc-amigaos/clib2 -name *.lo -exec rm -f {} +

find ./native-build/gcc-cross-build-11.3.0/ppc-amigaos/clib2 -name *.o -exec rm -f {} +

find ./native-build/gcc-cross-build-11.3.0/ppc-amigaos/clib2 -name *.so -exec rm -f {} +

find ./native-build/gcc-cross-build-11.3.0/ppc-amigaos/clib2 -name *.a -exec rm -f {} +



rm -f ./native-build/gcc-cross-build-11.3.0/ppc-amigaos/clib2/libstdc++-v3/src/.libs/*.*

rm -f ./native-build/gcc-cross-build-11.3.0/ppc-amigaos/clib2/libstdc++-v3/src/*.la

rm -f ./native-build/gcc-cross-build-11.3.0/ppc-amigaos/clib2/libstdc++-v3/src/*.lo

rm -f ./native-build/gcc-cross-build-11.3.0/ppc-amigaos/clib2/libstdc++-v3/src/*.o

rm -f ./native-build/gcc-cross-build-11.3.0/gcc/clib2/*.*





make -C native-build gcc-cross CROSS_PREFIX=/usr/local/amiga/

Just be sure that patches the same , i have it in /amiga/adtools, you may have it somewhere else. Also i install everything into /usr/local/amiga , you also may have some other paths.

Of course you also should have latest clib2 installed in SDK, so includes/etc will be taken from when recompilation happens.

After this all will be done, you can again try to recompile your test case (through i already tried - recompilation of libstdc++ indeed fix those linking issues).

Join us to improve dopus5!
AmigaOS4 on youtube

SinanSam460

Re: codecvt clib2 and newlib

Posted on: 2022/9/25 11:49 #7

Not too shy to talk

@kas1e

Thanks, I think I am on Gcc v11.1 :)

How can I update to V11.3 first ?

Sorry for these beginner questions :(

I plan to gather all these informations to a document in near future.

Sinan - AmigaOS4 Beta-Tester
- AmigaOne X5000
- AmigaOne A1222
- Sam460ex

kas1e

Re: codecvt clib2 and newlib

Posted on: 2022/9/25 11:58 #8

Home away from home

@Sinan
Not big deal to upgrate to 11.3, 11.1 fine enough too, just replace in my strings 11.3 on 11.1 in patches and all will be same

Join us to improve dopus5!
AmigaOS4 on youtube

afxgroup

Re: codecvt clib2 and newlib

Posted on: 2022/9/25 16:29 #9

Amigans Defender

c++ is different from c (library speaking) so you need to recompile libstdc++ with updated clib2 to avoid all problems.
However I have an open issue that is causing some problems if streams are opened from c++ too early. It happens on shared libraries all of the times, while with static libraries it happened only one time in my tests.
So you should be able to use c++ without any problem (after recompiling libstdc++ of course..)

i'm really tired...

Register To Post
	Top Previous Topic Next Topic

Currently Active Users Viewing This Thread: 1 ( 0 members and 1 Anonymous Users )