Hello All!

Built a GoldED+ that holds text as UTF-8 internally. Nothing changes on
the wire - it writes to echoes in whatever XLATEXPORT says, as before.

What it buys you:
- messages in different charsets in one echo are each shown in their
own, all at once - CP437, CP850, CP866, KOI8-R, ISO-8859, UTF-8
- on unix and macOS no more luit and no more separate screen session in
a legacy charset - it runs in the native UTF-8 terminal as it is
- in Windows Terminal and conhost a fullwidth character (CJK) takes the
two cells it is due instead of being squeezed into one and shifting
the rest of the line
- charset comes from the system, nothing to configure
- conversion goes through iconv; on Windows through the codepage API,
on OS/2 through ULS. No more collecting .chs tables for the pairs you
need: these know them all. DOS included - the build in the archive
carries GNU libiconv, statically linked

The first point deserves spelling out, because it is the whole reason
for the exercise. Text inside GoldED+ used to be single-byte, so
everything came down to the terminal's charset. A message tagged
CHRS: CP437 or CP850 had nowhere to go on a CP866 screen - CP866 has no
umlauts and no accented Latin at all. You got question marks or garbage instead, and German, Polish and French echoes read like a lottery. Same
story on a KOI8-R terminal. Now the screen is UTF-8, which has room for
all of it: a German message in CP850, a Russian one in CP866 and a
Polish one in CP852 sit side by side in one echo and all three come out
as written.

CHRS: IBMPC is sorted out too. It is an old, vague identifier: first it
meant CP437, then it came to mean "the codepage of the machine the
message was written on". It used to work by accident - nobody
configured an IBMPC table, the bytes went through untouched, and under
a matching terminal that was the right answer. Take IBMPC literally as
CP437 and a Russian message falls apart. It is now resolved properly:
the session charset when that is single-byte, and on a UTF-8 session
the DOS codepage the locale implies. A ^ACODEPAGE: line next to it wins
over IBMPC, as FTS-5003 requires.

CHRS: ASCII is taken strictly as seven-bit ISO 646-1, per the standard.
If a message claims ASCII and carries eight-bit bytes anyway, those
become question marks - but that message is mislabelled by its sender,
and the old asc_* tables did exactly the same thing.

Recompile your nodelist. The index held a name in 36 bytes, which was
36 characters while a character was a byte. In UTF-8 that is seventeen
Cyrillic letters, so a name like "Александр Христофоров" - 41 bytes -
no longer fits and got cut in the middle. The field is 80 now. That
changes the record format, GoldED+ will not touch an old .gxn - run
goldnode from the same archive.

Two new keywords, that is all:

XLATCONFIGSET - the charset GoldED+'s own files are written in:
golded.cfg, the language file, templates, tagline
files, the help file
XLATAREASET - the charset of area descriptions in your tosser's
area file. Unset, it follows XLATCONFIGSET

Both belong at the very top of the configuration, ahead of any line
whose value is not plain ASCII - values are converted as each line is
read. The usual case is a UTF-8 terminal with an older config and
language file:

XLATCONFIGSET CP866

There is also a GOLDED_CONSOLE environment variable (Windows only):
cells or stream, for when GoldED+ guesses wrong about how your console
draws a fullwidth character.

And if you do want to write in UTF-8, there are echoes for exactly
that - UTF-8 and UTF8.FTN.MESSAGING. Ask your uplink for them and
write in any language you like. I keep a group for them:

GROUP UTF-8;
MEMBER UTF-8, UTF8.FTN.MESSAGING
XLATIMPORT UTF-8
XLATEXPORT UTF-8
ENDGROUP

Everything else stays 8-bit as it was.

Built - fifteen of them, DOS, OS/2, Windows (MSVC6 included), Linux,
macOS, Solaris, Haiku:

https://github.com/evs38/golded-plus/releases

If you build your own:

git clone https://github.com/evs38/golded-plus.git

unicode is the default branch there. Builds as usual - cmake, or
make PLATFORM=lnx; see INSTALL and docs/building.txt.
On unix, curses and iconv are required.

Curses deserves a word: this is a transition period and it is easy to
get wrong. Wide-character support used to live in a separate library,
ncursesw, and in many systems that library no longer exists as such -
since ncurses 6 the wide calls sit in the ordinary ncurses, and
ncursesw is either a compatibility symlink to it or gone entirely.
pkgsrc, for one, has no separate ncursesw any more, only ncurses - and
it is the wide one. On older systems, though, they are still two
different packages, and there the wide one is exactly the one with the
"w".

The rule is simple: if your system has ncursesw, install that. If it
does not, the wide calls are already in the plain ncurses - install
that. The build then tells you what it found and whether it is wide.
iconv is part of libc on Linux; on the BSDs and macOS it is usually a
separate libiconv from packages.

On Windows neither is needed: the screen goes through its own console
layer and conversion through the codepage API. On OS/2 it is VIO and
ULS, both system. On DOS you need libiconv for djgpp if you want to
build with ICONV=1.

New build switches (make form below; cmake takes the same through -D,
with a GOLD_ prefix on the last two):

GOLD_UTF8=0 hold text single-byte, as before. Default is 1
everywhere except DOS
WIDE_NCURSES=0 the old 8-bit curses API instead of the wide one
EXTERNAL_CURSES=0 use only the system's curses, not a packaged one
EXTERNAL_ICONV=0 the same for iconv

By default on unix both come from the package manager (/usr/pkg,
/usr/local, /opt/homebrew, /opt/local, /opt/csw) with the system's own
behind them: a packaged ncurses is usually newer and wide-capable, and
a packaged libiconv knows more charsets.

On OS/2 conversion goes through ULS, the system's Unicode API. It is
picked up by itself when the toolkit headers are found; without them
you get a build on the tables. The DOS build in the archive is made
with ICONV=1 and libiconv for djgpp linked statically, so the .chs
tables are not needed there either; built without ICONV=1, DOS stays on
the tables as before.

There are bound to be plenty of bugs: GoldED+ is a huge construction
set grown over decades and nobody quite remembers all of its features
any more :) and almost everything that puts text on the screen was
touched here. So find bugs and write, I go through all of it.

* Originally in GOLDED
* Crossposted in UTF-8
* Crossposted in UTF8.FTN.MESSAGING
* Crossposted in FIDO_UTIL

Eugene

... It's full of stars!
--- GoldED+/BSD 1.1.5-b20260828 (NetBSD 11.0 Intel Core Haswell)
* Origin: FireFox Station (2:5075/35)

Hello Eugene,

Currently a reply always gets the area's (group's) XLATEXPORT, not the charset of the message being answered.
If someone writes me netmail in UTF-8, I still answer in CP850 (or CP866, whatever the group says). Ctrl-J does not change this, it only affects display.

How about a new parameter XlatReplyOriginal (True/False) to use in groups? :) So, on Reply/Quote take the original CHRS and use that for the answer.
New mail stays on XLATEXPORT. No CHRS on the original? Same, keep the default.

For example: netmail only, in a group:

GROUP NETMAIL
MEMBER NETMAIL
XLATEXPORT CP850
XlatReplyOriginal Yes
ENDGROUP

For new message in that group: CP850 will be used.
Reply to CHRS: UTF-8 4: UTF-8.
Reply to CHRS: CP866 2: CP866.

I believe that would be a great feature :)

Med vänlig hälsning / Best regards,
Alex.

---
* Origin: Rabarber > Skellefteå > Västerbotten > Sverige (2:203/910)

Hello Alexey!

28 Aug 26 21:13, you wrote to Eugene Subbotin:

How about a new parameter XlatReplyOriginal (True/False) to use in
groups? :) So, on Reply/Quote take the original CHRS and use that for the answer.
New mail stays on XLATEXPORT. No CHRS on the original? Same, keep the default.

How is it supposed to work when replying to another area? Say we're replying to a message from area A with XlatReplyOriginal On, to area B, where it's Off. Will the parameter apply?

I'm gonna add it to AmberEdit. ;-)

Yegor
--- AmberEdit/linux 0.5.1
* Origin: to err is human, but to really f.. things up you need AI (2:382/736)

Hello Yegor,

Friday August 28 2026 23:11, you wrote to me:

How about a new parameter XlatReplyOriginal (True/False) to use
in groups? :) So, on Reply/Quote take the original CHRS and use
that for the answer. New mail stays on XLATEXPORT. No CHRS on the
original? Same, keep the default.

How is it supposed to work when replying to another area? Say we're replying to a message from area A with XlatReplyOriginal On, to area
B, where it's Off. Will the parameter apply?

I believe it should follow the ruleset of the target area it that case.. And what's your vision on that?

I'm gonna add it to AmberEdit. ;-)

That should be quite useful :)

Mvh,
Alex.

---
* Origin: Rabarber > Skellefteå > Västerbotten > Sverige (2:203/910)

Hello Alexey!

28 Aug 26 23:25, you wrote to me:

How about a new parameter XlatReplyOriginal (True/False) to use
in groups? :) So, on Reply/Quote take the original CHRS and use
that for the answer. New mail stays on XLATEXPORT. No CHRS on the
original? Same, keep the default.

How is it supposed to work when replying to another area? Say we're
replying to a message from area A with XlatReplyOriginal On, to area
B, where it's Off. Will the parameter apply?

I believe it should follow the ruleset of the target area it that case.. And what's your vision on that?

Yeah, I'm also leaning toward the option with target area settings.

Yegor
--- AmberEdit/linux 0.5.1
* Origin: to err is human, but to really f.. things up you need AI (2:382/736)

Hello Alexey!

Friday August 28 2026 21:13, you wrote to me:

Currently a reply always gets the area's (group's) XLATEXPORT, not the charset of the message being answered. If someone writes me netmail in UTF-8, I still answer in CP850 (or CP866, whatever the group says).
Ctrl-J does not change this, it only affects display.

How about a new parameter XlatReplyOriginal (True/False) to use in
groups? :) So, on Reply/Quote take the original CHRS and use that for
the answer. New mail stays on XLATEXPORT. No CHRS on the original?
Same, keep the default.

XlatReplyOriginal is in the 20260829 build. It works globally and in a group, next to XlatImport and XlatExport:

GROUP NETMAIL;
MEMBER NETMAIL
XLATEXPORT CP850
XlatReplyOriginal Yes
ENDGROUP

A new message goes out CP850. A reply to UTF-8 goes UTF-8, a reply to CP866 goes CP866, and a message carrying no CHRS leaves CP850 alone. Replying or forwarding into another area follows that area's setting, and so do carbon copies and crossposts.

It took a good deal more work than the keyword deserved, and I think you will see why. The point of answering in his charset is that the man can certainly read you - and that only holds if the CHRS you put on the letter is one his reader recognises. Ours often was not. Pick ISO-8859-1 and we wrote "CHRS: ISO-8859-1 2", which is not a name FTS-5003 knows: there it is LATIN-1. ASCII went out at level 2 where the standard says 1. IBMPC, +7_FIDO and MAC must not go into a new message at all, and we sent them anyway. And +7_FIDO could not even be read: something that turns '_' back into a space had been applied to the one identifier that has an underscore of its own, so it arrived as "+7" and matched nothing at all.

So before "answer in his charset" could mean much, the labels had to start telling the truth. They do now. A reply to a seven-bit ASCII letter goes out in CP437, level 1 being kept by the standard only for reading old mail.

The export charset now can also be picked by hand while writing: Ctrl-Y in the reader, "Export Charsets" in the menu before the editor, "Change Export Charset" when it asks whether to save. The list opens on the charset in force - while answering, that is the one you are answering in - and Auto hands the choice back to XlatExport.

If you have time to bang on it: netmail in UTF-8, CP866, CP850 and one with no CHRS, a reply into another area and a forward, a carbon copy and a crosspost, which should take the charset of the area they land in, and picking a charset by hand, then Auto to undo it. Watch what lands in the CHRS line each time.

https://github.com/evs38/golded-plus/releases

There will be bugs :) Tell me what you hit.

Eugene

... It's full of stars!
--- GoldED+/BSD 1.1.5-b20260828 (NetBSD 11.0 Intel Core Haswell)
* Origin: FireFox Station (2:5075/35)

Hello Eugene,

Saturday August 29 2026 13:16, you wrote to me:

How about a new parameter XlatReplyOriginal (True/False) to use
in groups? :) So, on Reply/Quote take the original CHRS and use
that for the answer. New mail stays on XLATEXPORT. No CHRS on the
original? Same, keep the default.

XlatReplyOriginal is in the 20260829 build. It works globally and in a group, next to XlatImport and XlatExport:

A new message goes out CP850. A reply to UTF-8 goes UTF-8, a reply to CP866 goes CP866, and a message carrying no CHRS leaves CP850 alone. Replying or forwarding into another area follows that area's setting,
and so do carbon copies and crossposts.

It works exactly as expected! :)
I didn't think about those different cases with receiving messages of some outdated/incorrect CHRS in them, and aligning them to the FTS-5003.
Thanx, that is a big deal I can imagine.

If you have time to bang on it: netmail in UTF-8, CP866, CP850 and one with no CHRS, a reply into another area and a forward, a carbon copy
and a crosspost, which should take the charset of the area they land
in, and picking a charset by hand, then Auto to undo it. Watch what
lands in the CHRS line each time.

I made a number of tests with different combinations and no bugs so far!

Med vänlig hälsning / Best regards,
Alex.

---
* Origin: Rabarber > Skellefteå > Västerbotten > Sverige (2:203/910)

Hello Eugene,

On 29 Aug 26 13:16, you wrote to Alexey Matrosov:

XlatReplyOriginal is in the 20260829 build. It works globally and in a group, next to XlatImport and XlatExport:

GROUP NETMAIL;
MEMBER NETMAIL
XLATEXPORT CP850
XlatReplyOriginal Yes
ENDGROUP

20260829 Installed.

There will be bugs :) Tell me what you hit.

None found yet.


Cheers, Michiel

--- GoldED+/W32-MINGW 1.1.5-b20260829
* Origin: Klein Schnøørd (2:280/5555.1)

Hello Eugene,

On 29 Aug 26 13:16, you wrote to Alexey Matrosov:

The export charset now can also be picked by hand while writing:
Ctrl-Y in the reader, "Export Charsets" in the menu before the editor,

I still see the ISO-8859-x in the list of selectable character sets instead of the LATIN-x as is used in Fidonet.


Cheers, Michiel

--- GoldED+/W32-MINGW 1.1.5-b20260829
* Origin: Klein Schnøørd (2:280/5555.1)

Hello Michiel!

Saturday August 29 2026 15:56, you wrote to me:

The export charset now can also be picked by hand while writing:
Ctrl-Y in the reader, "Export Charsets" in the menu before the
editor,

MvdV> I still see the ISO-8859-x in the list of selectable character sets
MvdV> instead of the LATIN-x as is used in Fidonet.

It's ok. Official charset name is ISO-8859-x, but xlatexport fidonet kludge will be FTS-5003 compliant, like LATIN-1, LATIN-2, LATIN-5, etc.

Eugene

... It's full of stars!
--- GoldED+/BSD 1.1.5-b20260828 (NetBSD 11.0 Intel Core Haswell)
* Origin: FireFox Station (2:5075/35)

Hello Eugene,

On 29 Aug 26 16:59, you wrote to me:

MvdV>> I still see the ISO-8859-x in the list of selectable character
MvdV>> sets instead of the LATIN-x as is used in Fidonet.

It's ok. Official charset name is ISO-8859-x,

I know that - outside of Fidonet - they are the official names.

but xlatexport fidonet kludge will be FTS-5003 compliant, like
LATIN-1, LATIN-2, LATIN-5, etc.

Exacty! As a Fidonet sysop, that is how I know them and so that is how I would like to see it in that list.


Cheers, Michiel

--- GoldED+/W32-MINGW 1.1.5-b20260829
* Origin: Klein Schnøørd (2:280/5555.1)

Hello Eugene,

29 Aug 26 16:11, I wrote to you:

I know that - outside of Fidonet - they are the official names.

but xlatexport fidonet kludge will be FTS-5003 compliant, like
LATIN-1, LATIN-2, LATIN-5, etc.

Exacty! As a Fidonet sysop, that is how I know them and so that is how
I would like to see it in that list.

Also not that the numbers behind the dash to not correspond between the two naming conventions: From FTS-5003:

LATIN-1 ISO 8859-1 (Western European)
LATIN-2 ISO 8859-2 (Eastern European)
LATIN-5 ISO 8859-9 (Turkish)
LATIN-9 ISO 8859-15 (Western Europe with EURO sign)

So using the official names is confusing for fidonet sysops.


Cheers, Michiel

--- GoldED+/W32-MINGW 1.1.5-b20260829
* Origin: Klein Schnøørd (2:280/5555.1)

Hello Michiel!

Saturday August 29 2026 16:11, you wrote to me:

but xlatexport fidonet kludge will be FTS-5003 compliant, like
LATIN-1, LATIN-2, LATIN-5, etc.

MvdV> Exacty! As a Fidonet sysop, that is how I know them and so that is how
MvdV> I would like to see it in that list.

That's what was done in today's snapshot release. Please test it out.

Eugene

... It's full of stars!
--- GoldED+/BSD 1.1.5-b20260829 (NetBSD 11.0 Intel Core Haswell)
* Origin: FireFox Station (2:5075/35)

Hello Eugene,

On 30 Aug 26 09:26, you wrote to me:

but xlatexport fidonet kludge will be FTS-5003 compliant, like
LATIN-1, LATIN-2, LATIN-5, etc.

MvdV>> Exacty! As a Fidonet sysop, that is how I know them and so that
MvdV>> is how I would like to see it in that list.

That's what was done in today's snapshot release. Please test it out.

This what I see:


┌───── Charsets ──────┐
│ Auto │
│ UTF-8 -> UTF-8 │
│ CP866 -> UTF-8 │
│ CP1251 -> UTF-8 │
│ KOI8-R -> UTF-8 │
│ KOI8-U -> UTF-8 │
│ CP1125 -> UTF-8 │
│ CP855 -> UTF-8 │
│ ISO-8859-5 -> UTF-8 │
│ CP10007 -> UTF-8 │
│ CP437 -> UTF-8 │
│ CP850 -> UTF-8 │
│ CP852 -> UTF-8 │
│ CP858 -> UTF-8 │
│ CP1250 -> UTF-8 │
│ CP1252 -> UTF-8 │
│ LATIN-1 -> UTF-8 │
│ LATIN-2 -> UTF-8 │
│ LATIN-5 -> UTF-8 │
│ LATIN-9 -> UTF-8 │
│ CP10000 -> UTF-8 │
│ ASCII -> UTF-8 │ └─────────────────────┘

┌───── Charsets ──────┐
│ Auto │
│ UTF-8 <- UTF-8 │
│ CP866 <- UTF-8 │
│ CP1251 <- UTF-8 │
│ KOI8-R <- UTF-8 │
│ KOI8-U <- UTF-8 │
│ CP1125 <- UTF-8 │
│ CP855 <- UTF-8 │
│ ISO-8859-5 <- UTF-8 │
│ CP10007 <- UTF-8 │
│ CP437 <- UTF-8 │
│ CP850 <- UTF-8 │
│ CP852 <- UTF-8 │
│ CP858 <- UTF-8 │
│ CP1250 <- UTF-8 │ └─────────────────────┘

What is ISO-8859-5? And why is there no LATIN-x in the export list? And no CP1252 and CP10000?

I will probably never need it, just wondering why the import and export lists are different.


Cheers, Michiel

--- GoldED+/W32-MINGW 1.1.5-b20260830
* Origin: Nieuw Schnøørd (2:280/5555)

Hello Michiel!

Sunday August 30 2026 22:43, you wrote to me:

MvdV>>> Exacty! As a Fidonet sysop, that is how I know them and so
MvdV>>> that is how I would like to see it in that list.

That's what was done in today's snapshot release. Please test it
out.

MvdV> What is ISO-8859-5? And why is there no LATIN-x in the export list?
MvdV> And no CP1252 and CP10000?

This is a Cyrillic encoding that was previously used on the internet. Since GoldED can read not only Fidonet conferences but also newsgroups, it has been retained for compatibility.

MvdV> I will probably never need it, just wondering why the import and
MvdV> export lists are different.

This is strange. I couldn't reproduce it: both lists look identical. This raises the following questions:
1. Was the app window height the same when both lists were opened?
2. Does the shorter list scroll down (GoldED doesn't show a scrollbar in such windows)?
3. Are the conversion tables enabled via XlatCharset, and which ones?
4. Does this behavior persist if all XlatCharset and charsets.cfg files are commented out?

A message with a Cyrillic name in CP866 also didn't make it into my database, which means the issue isn't UTF-8, but is definitely related to the operation of husky hpt on BSD OS.


Eugene

... It's full of stars!
--- GoldED+/BSD 1.1.5-b20260830 (NetBSD 11.0 Intel Core Haswell)
* Origin: FireFox Station (2:5075/35)

Hello Eugene,

On 31 Aug 26 05:52, you wrote to me:

out.

MvdV>> What is ISO-8859-5? And why is there no LATIN-x in the export
MvdV>> list? And no CP1252 and CP10000?

This is a Cyrillic encoding that was previously used on the internet. Since GoldED can read not only Fidonet conferences but also
newsgroups, it has been retained for compatibility.

OK

MvdV>> I will probably never need it, just wondering why the import
MvdV>> and export lists are different.

This is strange. I couldn't reproduce it: both lists look identical.

Even more strange: now I cant'reproduce it either. Both lists are now identical except for direction of the arrows. :(

This raises the following questions: 1. Was the app window height the
same when both lists were opened? 2. Does the shorter list scroll down (GoldED doesn't show a scrollbar in such windows)?

AFAIR "no" to both.

3. Are the conversion tables enabled via XlatCharset, and which ones?

When I firdt installed Golded+ I commeted out all translattio tables in ythe config. Under the assumption they were not needed any more. Translation tables that for a large part I made myself and put quit a lot of hours in. But I am happy to let go of that in favour of somethin definitely better. ;-)

4. Does this behavior persist if all XlatCharset and charsets.cfg
files are commented out?

See above.

A message with a Cyrillic name in CP866 also didn't make it into my database, which means the issue isn't UTF-8, but is definitely related
to the operation of husky hpt on BSD OS.

Hmm...


Cheers, Michiel

--- GoldED+/W32-MINGW 1.1.5-b20260830
* Origin: Nieuw Schnøørd (2:280/5555)